feat(qualité): outillage de test complet, CI locale, annulation réelle des runs
Some checks are pending
CI / javascript (push) Waiting to run
CI / python (push) Waiting to run
CI / mutation (push) Waiting to run

Le dépôt n'avait aucun test, aucun linter, aucune vérification de types.

Outillage
- ESLint 9 (flat config) sur api + les deux frontends, Ruff sur le Python
- tsc --checkJs sur l'API (pas de TypeScript, juste la vérification)
- Vitest : 401 tests JS ; pytest : 43 tests Python
- Tests de mutation (Stryker), deux profils : logique pure et API complète
- Hook pre-push `npm run check` (~17 s) — le déploiement Coolify est sur webhook,
  c'est donc la seule porte de qualité avant la mise en ligne
- Workflow Forgejo Actions prêt (inerte tant qu'aucun runner n'est enregistré)

Sécurité
- Injection SQL authentifiée dans resolve-conflict : `field` était interpolé
  dans le SET sans allowlist
- timingSafeEqual levait sur un jeton multi-octets (500 au lieu de 401)
- setErrorHandler écrasait tous les 4xx en 500
- .env.example : ADMIN_JWT_SECRET et ADMIN_SEED_* n'étaient documentés nulle part
  alors que leur absence casse toute connexion admin

Annulation réelle des crawl_run (migration 014)
- L'API posait status='error' sans que le crawler en sache rien : le process
  continuait, et son UPDATE final ne matchait plus (run réussi affiché en erreur)
- Protocole coopératif : drapeau cancel_requested lu à chaque lot, le crawler
  écrit lui-même status='cancelled'

Cohérence géographique (migration 014)
- Le trigger 013 supprimait les band_locations sans purger le point dénormalisé
- L'édition admin de lat/lon n'atteignait jamais band_locations : la carte
  ignorait la correction. Override step_order = -1, dans une transaction

Corrections
- limit/offset NaN → 500 au lieu de 400
- OPTIONS sans `return reply` (Fastify poursuivait le cycle de vie)
- listen() sans catch, cast ::text en dur sur les colonnes numériques
- /admin/api/logs ne renvoyait pas sa pagination
- a11y : sélecteur de langue annoncé comme liste vide (role=option manquant)

Nettoyage
- apps/web/quizz-site supprimé (sans rapport avec le projet)
- Code mort : openModal(), LANG_NAMES, double import, variables inutilisées
- .dockerignore ajoutés ; node_modules racine n'était pas gitignoré

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Nicolas Fryder 2026-08-18 10:05:40 +02:00
parent a4c53fe09d
commit 60074fb015
58 changed files with 14084 additions and 3417 deletions

75
.forgejo/workflows/ci.yml Normal file
View file

@ -0,0 +1,75 @@
# CI informative.
#
# Le déploiement reste piloté par le webhook Coolify (push → redeploy), CE
# WORKFLOW NE BLOQUE RIEN : il marque simplement le commit vert ou rouge dans
# Forgejo. La vraie porte de qualité est le hook `.githooks/pre-push` côté
# poste de dev (voir README-CI.md).
#
# ⚠ Prérequis : un runner Forgejo Actions (act_runner) enregistré sur le VPS.
# Sans runner, ce fichier est inerte — aucun job ne démarre et aucun statut
# n'apparaît. C'est le cas aujourd'hui : ce workflow est là pour le jour où
# un runner sera ajouté.
name: CI
on:
push:
branches: [dev, main]
pull_request:
branches: [dev, main]
workflow_dispatch:
jobs:
javascript:
runs-on: docker
container:
image: node:22-alpine
steps:
- uses: actions/checkout@v4
- name: Installer les dépendances
run: npm ci
- name: ESLint
run: npm run lint
- name: Vérification de types (tsc --checkJs)
run: npm run typecheck
- name: Tests unitaires + a11y
run: npm run test
- name: Audit des dépendances
run: npm run audit:js
continue-on-error: true # informatif : ne doit pas masquer un échec de test
python:
runs-on: docker
container:
image: python:3.12-slim
steps:
- uses: actions/checkout@v4
- name: Installer l'outillage
run: pip install --no-cache-dir ruff pytest pip-audit
- name: Ruff (lint + règles de sécurité bandit)
run: ruff check .
- name: Tests du parseur de géocodage
run: cd apps/geocoder && python -m pytest tests -q
- name: Audit des dépendances
run: pip-audit -r apps/crawler/requirements.txt -r apps/geocoder/requirements.txt
continue-on-error: true
# Les tests de mutation prennent ~1 min : trop lents pour chaque push, mais
# utiles en revue. Déclenchement manuel uniquement.
mutation:
runs-on: docker
container:
image: node:22-alpine
if: github.event_name == 'workflow_dispatch'
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm run test:mutation

32
.githooks/pre-push Normal file
View file

@ -0,0 +1,32 @@
#!/bin/sh
#
# Porte de qualité locale — c'est ICI que la CI de ce projet se joue.
#
# Le déploiement Coolify se déclenche sur webhook à chaque push : rien ne
# s'interpose entre `git push` et la mise en ligne. Ce hook est donc le dernier
# filet avant que du code cassé parte en dev.
#
# Installation (une fois par clone) : npm run hooks:install
# Contournement ponctuel : git push --no-verify
#
set -e
echo "▶ pre-push : porte de qualité (lint, types, tests)…"
start=$(date +%s)
if ! npm run --silent check; then
cat >&2 <<'MSG'
✗ La porte de qualité a échoué — push interrompu.
Corriger, ou pousser quand même en connaissance de cause :
git push --no-verify
Correction automatique de la plupart des problèmes de style :
npm run lint:fix && npm run lint:py:fix
MSG
exit 1
fi
echo "✓ pre-push : tout est vert ($(( $(date +%s) - start ))s)"

16
.gitignore vendored
View file

@ -1,10 +1,24 @@
infra/.venv/ infra/.venv/
infra/temp/ infra/temp/
infra/geocode-enqueue.log infra/geocode-enqueue.log
infra/.env
# Node
node_modules/
apps/api/node_modules/ apps/api/node_modules/
# Python
__pycache__/ __pycache__/
*.pyc *.pyc
*.pyo *.pyo
infra/.env .pytest_cache/
.ruff_cache/
# Sorties d'outillage qualité
coverage/
reports/
.stryker-tmp/
# Session Claude Code (contient des secrets)
.claude/ .claude/
CLAUDE.md CLAUDE.md

134
README-CI.md Normal file
View file

@ -0,0 +1,134 @@
# Qualité, tests et CI/CD
## Le modèle en une phrase
Coolify redéploie sur webhook à **chaque** push (`dev` → dev.metalfrom.eu,
`main` → metalfrom.eu). Rien ne s'interpose. La porte de qualité est donc
**locale**, dans le hook `pre-push`.
```
git push ──▶ [hook pre-push : npm run check] ──▶ Forgejo ──▶ webhook ──▶ Coolify redeploy
↑ ~15 s, bloquant
```
## Installation (une fois par clone)
```bash
npm install
npm run hooks:install # active .githooks/pre-push
pip install -r requirements-dev.txt
```
## Commandes et temps mesurés
| Commande | Ce que ça fait | Froid | Incrémental |
|---|---|---|---|
| `npm run check` | **La porte** : ESLint + Ruff + tsc + tous les tests | ~15 s | — |
| `npm test` | 401 tests JS | 3,5 s | — |
| `npm run test:py` | 43 tests Python | 1 s | — |
| `npm run lint` / `lint:py` | ESLint / Ruff | 3 s / 1 s | — |
| `npm run typecheck` | `tsc --checkJs` sur l'API | 6 s | — |
| `npm run test:mutation` | Mutation, logique pure (352 mutants) | 34 s | ~5 s |
| `npm run test:mutation:full` | Mutation, toute l'API (1511 mutants) | 5 min | 13 s |
| `npm run check:full` | `check` + audits de dépendances + mutation complète | — | ~1 min |
### Où passe le temps, et pourquoi c'est acceptable
La commande de la boucle de développement, c'est `npm run check` : **17 s**,
et elle inclut déjà les 401 tests, ESLint, Ruff et tsc. C'est elle qui tourne
cent fois par jour.
La mutation n'est pas une commande de boucle courte. Son coût se lit ainsi :
- `test:mutation` (352 mutants) : 34 s à froid, ~5 s ensuite. Assez rapide pour
être lancée avant chaque commit qui touche la validation ou l'auth.
- `test:mutation:full` (1511 mutants) : 5 min **une seule fois** (clone neuf ou
après avoir modifié les quatre fichiers mutés d'un coup). Après une édition
normale d'un seul fichier : **13 s**, mesuré.
Le fichier incrémental vit dans `reports/`, qui est gitignoré : un clone neuf
paie donc les 5 minutes une fois. C'est assumé — c'est un audit, pas un test.
**Ce qui n'a délibérément pas été fait pour aller plus vite** : réduire encore
le périmètre muté. Descendre sous ~1500 mutants sur l'API reviendrait à ne plus
mesurer grand-chose, et un score de mutation flatteur obtenu en retirant les
mutants gênants est pire qu'une absence de score.
## Ce que couvrent les tests
| Batterie | Où | Approche |
|---|---|---|
| **Unitaires** | `apps/api/test/validate.test.js` | Validation pure : bbox, limit/offset, coordonnées, années, recherche |
| **Routes / intégration** | `publicRoutes*.test.js`, `adminRoutes*.test.js` | `fastify.inject()` + faux pool. Gating d'auth, allowlists de colonnes, forme du SQL, transactions |
| **Sécurité** | `adminAuth.test.js`, `rateLimit.test.js` | JWT forgé, `alg:none`, expiration, anti-énumération, verrouillage de compte, plafonds de débit par IP et par jeton |
| **Annulation** | `cancellation.test.js` + `apps/crawler/tests/test_cancellation.py` | Les deux moitiés du protocole coopératif |
| **Accessibilité** | `apps/web/test/a11y.test.js` | axe-core + jsdom sur le HTML statique |
| **Frontend** | `apps/web/test/pure.test.js` | Filtrage, tri, échappement HTML |
| **Méta** | `apps/api/test/harness.test.js` | Vérifie le faux pool lui-même |
| **Python** | `apps/geocoder/tests/`, `apps/crawler/tests/` | Parseur de localisations, annulation |
| **Mutation** | `stryker*.config.json` | 82 % (logique pure) / 65 % (API complète) |
## Performance : les décisions et leurs mesures
Le principe : **aucun test ne monte de conteneur, ne compile, ni ne touche le réseau.**
Quatre optimisations, toutes mesurées (les intuitions non vérifiées se sont
révélées fausses au moins une fois) :
1. **Une app Fastify par fichier de test, pas par cas.**
`buildServer()` coûte 14 ms. À raison d'une construction par `it()`, ces
14 ms étaient multipliés par ~12 000 exécutions pendant un run de mutation.
Les tests partagent une app et reprogramment un pool via `pool.reset()`
(`test/helpers/testApp.js`).
2. **Config Vitest dédiée à la mutation.**
Stryker recharge les fichiers de test à chaque mutant. En ne déclarant que
ceux qui couvrent le code muté (et surtout pas le test a11y, qui initialise
jsdom en ~2 s), le profil pur est passé de **1 min 23 à 34 s**.
3. **Mutants `StringLiteral` exclus.**
Les handlers sont à ~80 % du SQL en template literals. Muter le contenu
d'une chaîne SQL ne mesure rien. Les retirer a fait passer le profil complet
de 7 min 50 à 5 min — et le score de 42 % à 65 %, parce que le bruit
disparaissait du dénominateur.
4. **Plafonds de débit paramétrables.**
Conséquence du point 1 : une app partagée accumule l'état du limiteur. Les
valeurs réelles de production sont vérifiées séparément dans
`rateLimit.test.js`, pour que la paramétrisation ne crée pas d'angle mort.
Ce qui a été **essayé et rejeté sur mesure** : le pool Vitest `threads` avec
`isolate: false`, réputé plus rapide, donne **18,8 s contre 3,5 s** ici (la
mise en place de jsdom est pénalisée). Le défaut (`forks`) est conservé.
## Pourquoi deux profils de mutation
La mutation prouve qu'un test **échoue** quand le code casse — la couverture ne
dit que « cette ligne a été exécutée ».
- `test:mutation` (validate.js, adminAuth.js) : **82 %**. Logique pure, score
interprétable, assez rapide pour être lancé souvent.
- `test:mutation:full` (+ adminRoutes.js, app.js) : **65 %**. Les handlers de
route plafonnent structurellement plus bas — beaucoup de mutants portent sur
des messages de log ou des branches `catch` dont l'observable exact n'a pas
d'importance. Utile en audit, pas en boucle courte.
Les seuils d'échec (70 % et 55 %) sont des cliquets anti-régression, pas des
objectifs.
### Ce qui n'est pas couvert
- Règles a11y exigeant un moteur de rendu (contraste, cibles tactiles) : jsdom
ne calcule pas de styles. À compléter par un audit Lighthouse manuel.
- `migrate.js` : s'exécute au démarrage du conteneur et appelle `process.exit`.
- Crawler et workers de géocodage : seuls le parseur et l'annulation sont
couverts, le reste est de l'I/O réseau et base.
- `apps/web/site/pure.js` est testé mais **exclu de la mutation** : il est
chargé via `fs` + `vm` (comme le fait le navigateur), donc l'instrumentation
Stryker ne l'atteindrait pas et afficherait un score faussement parfait.
## Forgejo Actions
`.forgejo/workflows/ci.yml` existe mais **ne tourne pas** : aucun runner
`act_runner` n'est enregistré sur le VPS. Le fichier est prêt si tu en ajoutes
un. Il est volontairement informatif — il ne bloque pas le déploiement.

View file

@ -168,7 +168,7 @@ function splitGenreElements(genreRows) {
for (const { genre, total } of genreRows) { for (const { genre, total } of genreRows) {
if (!genre) continue; if (!genre) continue;
const cleaned = genre.replace(/\([^)]*\)/g, ""); const cleaned = genre.replace(/\([^)]*\)/g, "");
const tokens = cleaned.split(/[\/;,]+/); const tokens = cleaned.split(/[/;,]+/);
for (const tok of tokens) { for (const tok of tokens) {
const kw = tok.trim(); const kw = tok.trim();
if (kw.length < 3) continue; if (kw.length < 3) continue;
@ -255,15 +255,23 @@ async function loadOverviewLive() {
const runsHtml = live.active_runs.length const runsHtml = live.active_runs.length
? live.active_runs.map(r => { ? live.active_runs.map(r => {
const elapsed = Math.round((Date.now() - new Date(r.started_at)) / 60000); const elapsed = Math.round((Date.now() - new Date(r.started_at)) / 60000);
// L'annulation est coopérative : le crawler ne s'arrête qu'au prochain
// point de contrôle. Tant qu'il n'a pas basculé le statut à
// 'cancelled', on affiche l'état intermédiaire au lieu de laisser
// croire que c'est déjà fait.
const pendingCancel = r.cancel_requested === true;
return `<div class="mon-run-row"> return `<div class="mon-run-row">
<div> <div>
<span class="badge warn"> ${esc(r.run_type)}</span> <span class="badge ${pendingCancel ? "muted" : "warn"}"> ${esc(r.run_type)}</span>
<div style="font-size:11px;color:var(--muted);margin-top:5px"> <div style="font-size:11px;color:var(--muted);margin-top:5px">
${elapsed}min · ${r.bands_seen} vus · ${r.bands_new} nouveaux · ${r.bands_enriched} enrichis ${elapsed}min · ${r.bands_seen} vus · ${r.bands_new} nouveaux · ${r.bands_enriched} enrichis
</div> </div>
${pendingCancel ? `<div style="font-size:11px;color:var(--warn);margin-top:3px">⏳ Arrêt demandé${r.cancel_requested_by ? ` par ${esc(r.cancel_requested_by)}` : ""} — le crawler s'arrêtera au prochain lot</div>` : ""}
${r.error ? `<div style="font-size:11px;color:var(--err);margin-top:3px">${esc(r.error)}</div>` : ""} ${r.error ? `<div style="font-size:11px;color:var(--err);margin-top:3px">${esc(r.error)}</div>` : ""}
</div> </div>
<button class="btn btn-mini" style="border-color:rgba(198,26,26,0.4)" onclick="adminCancelRun(${r.id})"> Annuler</button> ${pendingCancel
? `<button class="btn btn-mini" disabled title="Annulation déjà demandée">⏳ Arrêt…</button>`
: `<button class="btn btn-mini" style="border-color:rgba(198,26,26,0.4)" onclick="adminCancelRun(${r.id})">✕ Annuler</button>`}
</div>`; </div>`;
}).join("") }).join("")
: `<div class="empty" style="padding:16px 0">Aucun run actif</div>`; : `<div class="empty" style="padding:16px 0">Aucun run actif</div>`;
@ -341,7 +349,7 @@ async function loadOverviewLive() {
} }
async function adminCancelRun(id) { async function adminCancelRun(id) {
if (!confirm(`Annuler le run #${id} ?`)) return; if (!confirm(`Demander l'arrêt du run #${id} ?\n\nLe crawler s'arrêtera à son prochain point de contrôle (quelques secondes à une minute selon le job).`)) return;
try { try {
await api(`/admin/api/crawl-runs/${id}/cancel`, { method: "POST", body: "{}" }); await api(`/admin/api/crawl-runs/${id}/cancel`, { method: "POST", body: "{}" });
await loadOverviewLive(); await loadOverviewLive();
@ -356,6 +364,12 @@ async function adminCancelJob(id) {
} catch (e) { alert(`Erreur : ${e.message}`); } } catch (e) { alert(`Erreur : ${e.message}`); }
} }
// Ces deux handlers sont appelés depuis des attributs onclick= générés dans
// les templates ci-dessus : l'exposition sur `window` est explicite pour que le
// couplage reste visible (et que le linter ne les prenne pas pour du code mort).
window.adminCancelRun = adminCancelRun;
window.adminCancelJob = adminCancelJob;
// ------------------------------------------------------------------ // ------------------------------------------------------------------
// Groupes (Bands + Conflits fondus) // Groupes (Bands + Conflits fondus)
// ------------------------------------------------------------------ // ------------------------------------------------------------------

4
apps/api/.dockerignore Normal file
View file

@ -0,0 +1,4 @@
node_modules/
test/
coverage/
.stryker-tmp/

View file

@ -0,0 +1,78 @@
-- 014_run_cancellation_and_geo_sync.sql
--
-- Deux corrections d'invariants qui rendaient des fonctionnalités cosmétiques.
--
-- A) Annulation réelle des crawl_run
-- Avant : « Annuler » écrivait status='error' en base. Le crawler Python ne
-- lisait jamais cette colonne et continuait sa boucle jusqu'au bout. Pire,
-- son UPDATE final `WHERE id=%s AND status='running'` ne matchait plus, donc
-- un run mené à terme restait affiché en erreur. Le bouton ne faisait rien
-- d'autre que mentir.
-- Après : l'admin pose un DRAPEAU (cancel_requested) sans toucher au statut.
-- Le crawler le lit à chaque itération et s'arrête proprement, en écrivant
-- lui-même status='cancelled'. Le statut reflète donc toujours la réalité du
-- process.
--
-- B) Désynchronisation du point principal
-- bands.lat/lon/geom est une dénormalisation du PREMIER lieu géocodé de
-- band_locations (voir _sync_band_point dans geocoder/worker.py). Le trigger
-- de la migration 013 supprime les band_locations quand location_text change,
-- mais laissait l'ancien point sur bands : la carte (qui lit band_locations)
-- n'affichait plus rien tandis que la liste et la heatmap (qui lisent bands)
-- continuaient de montrer l'ancienne ville, parfois pendant des jours.
-- ------------------------------------------------------------------
-- A) Annulation coopérative
-- ------------------------------------------------------------------
ALTER TABLE crawl_run ADD COLUMN IF NOT EXISTS cancel_requested BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE crawl_run ADD COLUMN IF NOT EXISTS cancel_requested_at TIMESTAMPTZ;
ALTER TABLE crawl_run ADD COLUMN IF NOT EXISTS cancel_requested_by TEXT;
-- Le crawler interroge ce drapeau en boucle : index partiel pour que le
-- lookup reste un index-only scan même avec un historique de runs volumineux.
CREATE INDEX IF NOT EXISTS idx_crawl_run_cancel
ON crawl_run (id)
WHERE status = 'running' AND cancel_requested = TRUE;
-- Note : on n'ajoute délibérément PAS de cancel_requested sur job_triggers.
-- Un job 'pending' se retire de la file (c'est ce que fait la route existante) ;
-- une fois réclamé par le crawler il n'existe plus qu'à travers le crawl_run
-- qu'il a créé, et c'est CE run qu'on annule. Deux drapeaux pour un seul état
-- réel, c'est exactement le genre de colonne qui finit morte.
-- ------------------------------------------------------------------
-- B) Le point principal suit la suppression des localisations
-- ------------------------------------------------------------------
CREATE OR REPLACE FUNCTION bands_geocode_dirty() RETURNS trigger AS $$
BEGIN
IF TG_OP = 'UPDATE' AND OLD.location_text IS DISTINCT FROM NEW.location_text THEN
DELETE FROM band_locations WHERE ma_id = NEW.ma_id;
-- Purge aussi le point dénormalisé : il décrivait l'ancienne localisation.
-- Cet UPDATE ne touche pas location_text, donc il ne redéclenche pas ce
-- trigger (défini AFTER UPDATE OF location_text) — pas de récursion.
UPDATE bands
SET lat = NULL,
lon = NULL,
geocoded_at = NULL,
geocode_provider = NULL,
geocode_query = NULL
WHERE ma_id = NEW.ma_id
AND (lat IS NOT NULL OR lon IS NOT NULL);
-- geom est recalculé (à NULL) par le trigger BEFORE bands_set_geom.
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- ------------------------------------------------------------------
-- C) Localisation posée manuellement par un admin
-- ------------------------------------------------------------------
-- Éditer lat/lon depuis le dashboard n'écrivait que sur bands : la carte, qui
-- clusterise depuis band_locations, ignorait totalement la correction. On
-- réserve step_order = -1 à l'override manuel, qui passe donc avant tous les
-- steps issus du parsing (tri step_order ASC) et devient le point principal.
COMMENT ON COLUMN band_locations.step_order IS
'Ordre de l''étape dans location_text (0 = première). -1 = override manuel admin, prioritaire.';

View file

@ -1,4 +1,5 @@
import { requireAdminSession, writeAuditLog } from "./adminAuth.js"; import { requireAdminSession, writeAuditLog } from "./adminAuth.js";
import { pagination } from "./validate.js";
const BAND_SORT_COLUMNS = new Set([ const BAND_SORT_COLUMNS = new Set([
"ma_id", "name", "country", "status", "genre", "ma_id", "name", "country", "status", "genre",
@ -10,11 +11,8 @@ const BAND_EDITABLE_FIELDS = [
"themes", "location_text", "lat", "lon", "themes", "location_text", "lat", "lon",
]; ];
function pagination(query) { /** Sous-ensemble de BAND_EDITABLE_FIELDS stocke en colonne numerique. */
const page = Math.max(1, Number(query.page) || 1); const NUMERIC_BAND_FIELDS = new Set(["formed_year", "lat", "lon"]);
const pageSize = Math.min(200, Math.max(1, Number(query.pageSize) || 50));
return { page, pageSize, offset: (page - 1) * pageSize };
}
export default async function adminRoutes(fastify, opts) { export default async function adminRoutes(fastify, opts) {
const { pool } = opts; const { pool } = opts;
@ -268,34 +266,83 @@ export default async function adminRoutes(fastify, opts) {
updates.lon = v; updates.lon = v;
} }
const before = await pool.query(`SELECT * FROM bands WHERE ma_id = $1 LIMIT 1`, [id]); // Transaction : l'edition, la pose de l'override de localisation et
if (!before.rows.length) return reply.code(404).send({ ok: false, error: "not found" }); // l'ecriture d'audit doivent reussir ou echouer ensemble. Sans ca, une
// coupure entre les deux laissait une modification sans trace d'audit.
const client = await pool.connect();
try {
await client.query("BEGIN");
const setCols = Object.keys(updates); const before = await client.query(`SELECT * FROM bands WHERE ma_id = $1 LIMIT 1`, [id]);
// Marque les champs édités comme verrouillés (ne seront plus écrasés par le crawler) if (!before.rows.length) {
const newLocked = Object.fromEntries(setCols.map((c) => [c, true])); await client.query("ROLLBACK");
const setSql = setCols.map((c, idx) => `${c} = $${idx + 2}`).join(", "); return reply.code(404).send({ ok: false, error: "not found" });
const vals = [id, ...setCols.map((c) => updates[c])]; }
const after = await pool.query( const setCols = Object.keys(updates);
`UPDATE bands SET ${setSql}, // Marque les champs edites comme verrouilles (plus ecrases par le crawler)
locked_fields = locked_fields || $${vals.length + 1}::jsonb, const newLocked = Object.fromEntries(setCols.map((c) => [c, true]));
crawler_pending = ( const setSql = setCols.map((c, idx) => `${c} = $${idx + 2}`).join(", ");
SELECT jsonb_strip_nulls(jsonb_build_object(${ const vals = [id, ...setCols.map((c) => updates[c])];
setCols.map((c) => `'${c}', crawler_pending->'${c}'`).join(", ")
}))
FROM bands WHERE ma_id = $1
)
WHERE ma_id = $1 RETURNING *`,
[...vals, JSON.stringify(newLocked)]
);
await writeAuditLog( const after = await client.query(
pool, req.adminUsername, "update", "bands", id, `UPDATE bands SET ${setSql},
before.rows[0], after.rows[0] locked_fields = locked_fields || $${vals.length + 1}::jsonb,
); crawler_pending = (
SELECT jsonb_strip_nulls(jsonb_build_object(${
setCols.map((c) => `'${c}', crawler_pending->'${c}'`).join(", ")
}))
FROM bands WHERE ma_id = $1
)
WHERE ma_id = $1 RETURNING *`,
[...vals, JSON.stringify(newLocked)]
);
return { ok: true, item: after.rows[0] }; // La carte clusterise depuis band_locations, jamais depuis bands.
// Corriger lat/lon ici ne deplacait donc aucun point : il faut poser un
// override dans band_locations. step_order = -1 le place avant tous les
// steps issus du parsing, ce qui en fait aussi le point principal.
const row = after.rows[0];
if ("lat" in updates || "lon" in updates) {
if (row.lat != null && row.lon != null) {
await client.query(
`INSERT INTO band_locations
(ma_id, step_order, step_label, location_raw, is_country_only,
lat, lon, geocode_status, geocode_provider, geocode_confidence, updated_at)
VALUES ($1, -1, 'override admin', $2, FALSE, $3, $4, 'done', 'admin', 1.0, now())
ON CONFLICT (ma_id, step_order, location_raw) DO UPDATE SET
lat = EXCLUDED.lat,
lon = EXCLUDED.lon,
geocode_status = 'done',
geocode_provider = 'admin',
geocode_confidence = 1.0,
geocode_error = NULL,
updated_at = now()`,
[id, row.location_text || "(coordonnees saisies manuellement)", row.lat, row.lon]
);
} else {
// lat/lon effaces -> l'override n'a plus lieu d'etre.
await client.query(
`DELETE FROM band_locations WHERE ma_id = $1 AND step_order = -1`, [id]
);
}
}
await client.query(
`INSERT INTO admin_audit_log (admin_username, action, target_table, target_id, before_data, after_data)
VALUES ($1, $2, $3, $4, $5, $6)`,
[req.adminUsername, "update", "bands", String(id),
JSON.stringify(before.rows[0]), JSON.stringify(row)]
);
await client.query("COMMIT");
return { ok: true, item: row };
} catch (err) {
await client.query("ROLLBACK").catch(() => {});
throw err;
} finally {
client.release();
}
} catch (err) { } catch (err) {
fastify.log.error(err); fastify.log.error(err);
return reply.code(500).send({ ok: false, error: "Erreur mise à jour band" }); return reply.code(500).send({ ok: false, error: "Erreur mise à jour band" });
@ -341,7 +388,9 @@ export default async function adminRoutes(fastify, opts) {
LIMIT $${i} OFFSET $${i + 1} LIMIT $${i} OFFSET $${i + 1}
`, vals); `, vals);
return { ok: true, items: r.rows }; // page/pageSize etaient calcules mais jamais renvoyes : le client n'avait
// aucun moyen de savoir s'il restait des logs a charger.
return { ok: true, items: r.rows, page, pageSize };
} catch (err) { } catch (err) {
fastify.log.error(err); fastify.log.error(err);
return reply.code(500).send({ ok: false, error: "Erreur logs" }); return reply.code(500).send({ ok: false, error: "Erreur logs" });
@ -385,6 +434,12 @@ export default async function adminRoutes(fastify, opts) {
if (!field || !["keep_mine", "accept_crawler"].includes(action)) { if (!field || !["keep_mine", "accept_crawler"].includes(action)) {
return reply.code(400).send({ ok: false, error: "field et action requis" }); return reply.code(400).send({ ok: false, error: "field et action requis" });
} }
// `field` est interpole dans le SET plus bas (impossible de parametrer un
// nom de colonne) : sans cette allowlist, un admin authentifie pouvait
// ecrire du SQL arbitraire via le corps de la requete.
if (!BAND_EDITABLE_FIELDS.includes(field)) {
return reply.code(400).send({ ok: false, error: `field non modifiable: ${field}` });
}
const before = await pool.query(`SELECT * FROM bands WHERE ma_id = $1`, [id]); const before = await pool.query(`SELECT * FROM bands WHERE ma_id = $1`, [id]);
if (!before.rows.length) return reply.code(404).send({ ok: false, error: "not found" }); if (!before.rows.length) return reply.code(404).send({ ok: false, error: "not found" });
@ -396,7 +451,12 @@ export default async function adminRoutes(fastify, opts) {
} else { } else {
// Applique la valeur du crawler, déverrouille le champ // Applique la valeur du crawler, déverrouille le champ
const crawlerVal = before.rows[0].crawler_pending?.[field]; const crawlerVal = before.rows[0].crawler_pending?.[field];
sql = `UPDATE bands SET ${field} = $2::text, locked_fields = locked_fields - $3, crawler_pending = crawler_pending - $3 WHERE ma_id = $1`; // Le cast depend du type reel de la colonne : `::text` en dur cassait
// formed_year / lat / lon (erreur Postgres 42804).
const cast = NUMERIC_BAND_FIELDS.has(field)
? (field === "formed_year" ? "::int" : "::double precision")
: "::text";
sql = `UPDATE bands SET ${field} = $2${cast}, locked_fields = locked_fields - $3, crawler_pending = crawler_pending - $3 WHERE ma_id = $1`;
vals = [id, crawlerVal, field]; vals = [id, crawlerVal, field];
} }
const after = await pool.query(sql + " RETURNING *", vals); const after = await pool.query(sql + " RETURNING *", vals);
@ -663,7 +723,8 @@ export default async function adminRoutes(fastify, opts) {
const [active_runs, pending_jobs, geo_queue, geo_processing, checkpoints] = await Promise.all([ const [active_runs, pending_jobs, geo_queue, geo_processing, checkpoints] = await Promise.all([
pool.query(` pool.query(`
SELECT id, run_type, countries, status, started_at, SELECT id, run_type, countries, status, started_at,
bands_seen, bands_new, bands_updated, bands_enriched, error bands_seen, bands_new, bands_updated, bands_enriched, error,
cancel_requested, cancel_requested_at, cancel_requested_by
FROM crawl_run WHERE status = 'running' ORDER BY started_at DESC FROM crawl_run WHERE status = 'running' ORDER BY started_at DESC
`), `),
pool.query(` pool.query(`
@ -700,16 +761,40 @@ export default async function adminRoutes(fastify, opts) {
try { try {
const id = Number(req.params.id); const id = Number(req.params.id);
if (!Number.isFinite(id)) return reply.code(400).send({ ok: false, error: "bad id" }); if (!Number.isFinite(id)) return reply.code(400).send({ ok: false, error: "bad id" });
// Annulation COOPERATIVE : on pose un drapeau et on laisse le statut a
// 'running'. Le crawler le lit a chaque lot et ecrit lui-meme
// status='cancelled' quand il s'est reellement arrete.
//
// Ecrire directement status='error' ici (l'ancien comportement) mentait
// deux fois : le process continuait a tourner, et son UPDATE final
// `WHERE status='running'` ne matchait plus.
const r = await pool.query(` const r = await pool.query(`
UPDATE crawl_run SET status='error', finished_at=now(), error='annulé manuellement par admin' UPDATE crawl_run
WHERE id=$1 AND status='running' RETURNING id, run_type SET cancel_requested = TRUE,
`, [id]); cancel_requested_at = now(),
if (!r.rows.length) return reply.code(404).send({ ok: false, error: "Run non trouvé ou déjà terminé" }); cancel_requested_by = $2
WHERE id = $1 AND status = 'running' AND cancel_requested = FALSE
RETURNING id, run_type
`, [id, req.adminUsername]);
if (!r.rows.length) {
// Distinguer << deja demande >> de << run inexistant/termine >> : sinon
// double-cliquer sur Annuler affiche une erreur alarmante a tort.
const existing = await pool.query(
`SELECT status, cancel_requested FROM crawl_run WHERE id = $1`, [id]
);
if (existing.rows.length && existing.rows[0].cancel_requested) {
return { ok: true, id, already_requested: true };
}
return reply.code(404).send({ ok: false, error: "Run non trouvé ou déjà terminé" });
}
await pool.query( await pool.query(
`INSERT INTO crawl_log (level, message) VALUES ($1, $2)`, `INSERT INTO crawl_log (level, message) VALUES ($1, $2)`,
["warning", `[admin:${req.adminUsername}] run #${id} (${r.rows[0].run_type}) annulé manuellement`] ["warning", `[admin:${req.adminUsername}] annulation demandee pour le run #${id} (${r.rows[0].run_type})`]
).catch(() => {}); ).catch(() => {});
return { ok: true, id }; await writeAuditLog(pool, req.adminUsername, "cancel_run", "crawl_run", id, null, { run_type: r.rows[0].run_type });
return { ok: true, id, cancel_requested: true };
} catch (err) { } catch (err) {
fastify.log.error(err); fastify.log.error(err);
return reply.code(500).send({ ok: false, error: "Erreur annulation run" }); return reply.code(500).send({ ok: false, error: "Erreur annulation run" });
@ -727,7 +812,15 @@ export default async function adminRoutes(fastify, opts) {
UPDATE job_triggers SET status='cancelled', finished_at=now() UPDATE job_triggers SET status='cancelled', finished_at=now()
WHERE id=$1 AND status='pending' RETURNING id, job_type WHERE id=$1 AND status='pending' RETURNING id, job_type
`, [id]); `, [id]);
if (!r.rows.length) return reply.code(404).send({ ok: false, error: "Job non trouvé ou déjà traité" }); if (!r.rows.length) {
// Un job deja reclame par le crawler n'est plus annulable en tant que
// job : il n'existe plus que sous la forme du crawl_run qu'il a cree.
const existing = await pool.query(`SELECT status FROM job_triggers WHERE id = $1`, [id]);
const msg = existing.rows[0]?.status === "running"
? "Job déjà démarré — annuler le run correspondant dans Vue d'ensemble"
: "Job non trouvé ou déjà traité";
return reply.code(409).send({ ok: false, error: msg });
}
await pool.query( await pool.query(
`INSERT INTO crawl_log (level, message) VALUES ($1, $2)`, `INSERT INTO crawl_log (level, message) VALUES ($1, $2)`,
["warning", `[admin:${req.adminUsername}] job trigger #${id} (${r.rows[0].job_type}) annulé`] ["warning", `[admin:${req.adminUsername}] job trigger #${id} (${r.rows[0].job_type}) annulé`]

843
apps/api/src/app.js Normal file
View file

@ -0,0 +1,843 @@
/**
* Construction de l'application Fastify.
*
* Séparé de server.js (qui se contente d'écouter) pour que les tests puissent
* instancier l'app avec un faux pool et l'interroger via `fastify.inject()`,
* sans ouvrir de port ni de connexion Postgres.
*/
import Fastify from "fastify";
import rateLimit from "@fastify/rate-limit";
import helmet from "@fastify/helmet";
import cookie from "@fastify/cookie";
import { timingSafeEqual } from "crypto";
import {
ADMIN_COOKIE_NAME,
seedAdminUser,
signAdminSession,
isLockedOut,
recordLoginAttempt,
verifyPassword,
markLoginSuccess,
requireAdminSession,
} from "./adminAuth.js";
import adminApiRoutes from "./adminRoutes.js";
import { ValidationError, sanitizeSearchString, parseLimitOffset } from "./validate.js";
/**
* @param {object} [opts]
* @param {object|null} [opts.pool] pool pg (ou un double de test)
* @param {boolean} [opts.logger]
* @param {string} [opts.importToken] jeton Bearer de /admin/import
* @param {string[]} [opts.corsOrigins]
* @param {boolean} [opts.seedAdmin] false en test (évite un INSERT parasite)
* @param {number} [opts.globalRateLimitMax] requêtes/min, toutes routes
* @param {number} [opts.adminRateLimitMax] requêtes/min sur /admin/import
* @param {number} [opts.authRateLimitMax] requêtes/min sur /admin/auth/login
*/
export async function buildServer(opts = {}) {
const {
pool = null,
logger = false,
importToken = (process.env.BM_IMPORT_TOKEN || "").trim(),
corsOrigins = process.env.CORS_ORIGINS
? process.env.CORS_ORIGINS.split(",").map((s) => s.trim())
: ["https://metalfrom.eu", "https://www.metalfrom.eu"],
seedAdmin = true,
// Plafonds de débit paramétrables : les tests partagent une même instance
// d'app entre les cas (construire une app coûte ~14 ms, multipliées par des
// milliers d'exécutions en tests de mutation). Sans ça, l'état du limiteur
// s'accumulerait d'un test à l'autre et ferait échouer le 11e. Les valeurs
// par défaut restent celles de production, et sont testées explicitement
// dans apps/api/test/rateLimit.test.js.
globalRateLimitMax = 1000,
adminRateLimitMax = 10,
authRateLimitMax = 10,
} = opts;
const fastify = Fastify({
logger,
bodyLimit: 10485760, // 10 MB max pour éviter DoS mémoire
trustProxy: true // derrière Traefik (+ nginx pour /admin/*) : lit X-Forwarded-For
});
// Headers de sécurité
await fastify.register(helmet, {
contentSecurityPolicy: false,
crossOriginEmbedderPolicy: false
});
await fastify.register(cookie);
// CORS middleware
fastify.addHook('onRequest', async (request, reply) => {
const origin = request.headers.origin;
const allowedOrigins = corsOrigins;
if (allowedOrigins.includes(origin)) {
reply.header('Access-Control-Allow-Origin', origin);
}
reply.header('Access-Control-Allow-Methods', 'GET, POST, PUT, PATCH, DELETE, OPTIONS');
reply.header('Access-Control-Allow-Headers', 'Content-Type, Authorization');
reply.header('Access-Control-Allow-Credentials', 'true');
if (request.method === 'OPTIONS') {
// `return reply` est obligatoire : sans ça Fastify poursuit le cycle de vie
// et route quand même la requête OPTIONS vers un handler (ou un 404).
return reply.status(204).send();
}
});
// Rate limiting global
await fastify.register(rateLimit, {
max: globalRateLimitMax,
timeWindow: '1 minute',
cache: 10000
});
// Gestion d'erreurs globale
fastify.setErrorHandler((error, request, reply) => {
// Les erreurs 4xx (validation de schéma, rate-limit, payload trop gros…) ont
// un statusCode utile : l'écraser en 500 masquait la cause côté client et
// faisait passer des erreurs d'entrée pour des pannes serveur.
const status = Number(error.statusCode);
if (Number.isInteger(status) && status >= 400 && status < 500) {
return reply.status(status).send({ ok: false, error: error.message });
}
fastify.log.error(error);
return reply.status(500).send({
ok: false,
error: 'Une erreur est survenue'
});
});
if (pool && seedAdmin) {
seedAdminUser(pool).catch((err) => fastify.log.error({ err }, "[admin] seed failed"));
}
function requirePool() {
if (!pool) throw new Error("DATABASE_URL not set");
return pool;
}
function authBearer(req) {
const h = req.headers.authorization || "";
const m = h.match(/^Bearer\s+(.+)$/i);
return m ? m[1] : null;
}
function requireAdmin(req, reply) {
const provided = authBearer(req);
if (!importToken || !provided) {
reply.code(401).send({ ok: false, error: "unauthorized" });
return false;
}
// Comparer les longueurs en OCTETS, pas en caractères : timingSafeEqual
// lève si les deux buffers diffèrent en taille, et un jeton multi-octets
// ("é" = 2 octets) passait le test de longueur de chaîne tout en produisant
// des buffers de tailles différentes → exception → 500 au lieu de 401.
const providedBuf = Buffer.from(provided, "utf8");
const expectedBuf = Buffer.from(importToken, "utf8");
if (providedBuf.length !== expectedBuf.length) {
reply.code(401).send({ ok: false, error: "unauthorized" });
return false;
}
const valid = timingSafeEqual(providedBuf, expectedBuf);
if (!valid) {
reply.code(401).send({ ok: false, error: "unauthorized" });
return false;
}
return true;
}
fastify.get("/", async () => {
return { ok: true, service: "BM API" };
});
fastify.get("/api/health", async () => ({ ok: true }));
fastify.get("/api/db", async (req, reply) => {
try {
const p = requirePool();
const r = await p.query("SELECT now() as now, current_database() as db");
return { ok: true, ...r.rows[0] };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur de connexion à la base de données' });
}
});
fastify.get("/api/stats", async (req, reply) => {
try {
const p = requirePool();
const q = `
SELECT
count(*)::int as total,
count(*) FILTER (WHERE geom IS NOT NULL)::int as geocoded,
count(*) FILTER (WHERE geom IS NULL)::int as no_location,
count(*) FILTER (WHERE enriched = true)::int as enriched
FROM bands;
`;
const r = await p.query(q);
return { ok: true, ...r.rows[0] };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des stats' });
}
});
fastify.get("/api/countries", async (req, reply) => {
try {
const p = requirePool();
const q = `
SELECT
country,
count(*)::int as total,
count(*) FILTER (WHERE geom IS NOT NULL)::int as geocoded,
count(*) FILTER (WHERE enriched = true)::int as enriched
FROM bands
GROUP BY country
ORDER BY total DESC;
`;
const r = await p.query(q);
return { ok: true, items: r.rows };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des pays' });
}
});
fastify.get("/api/statuses", async (req, reply) => {
try {
const p = requirePool();
const q = `
SELECT
COALESCE(NULLIF(trim(status), ''), 'Unknown') as status,
count(*)::int as total,
count(*) FILTER (WHERE geom IS NOT NULL)::int as geocoded
FROM bands
GROUP BY 1
ORDER BY total DESC;
`;
const r = await p.query(q);
return { ok: true, items: r.rows };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des statuts' });
}
});
fastify.get("/api/clusters", async (req, reply) => {
try {
const p = requirePool();
const {
bbox,
zoom,
countries,
status,
genre,
year_min,
year_max,
} = /** @type {Record<string, string|undefined>} */ (req.query || {});
if (!bbox || !zoom) {
return reply.code(400).send({ ok: false, error: "bbox and zoom are required" });
}
const bboxParts = String(bbox).split(",").map(Number);
if (bboxParts.length !== 4) {
return reply.code(400).send({ ok: false, error: "bbox must have 4 values" });
}
const [minLon, minLat, maxLon, maxLat] = bboxParts;
const zoomLevel = Number(zoom);
if ([minLon, minLat, maxLon, maxLat, zoomLevel].some(v => !Number.isFinite(v))) {
return reply.code(400).send({ ok: false, error: "Invalid bbox or zoom values" });
}
if (minLat < -90 || maxLat > 90 || minLon < -180 || maxLon > 180) {
return reply.code(400).send({ ok: false, error: "Coordinates out of range" });
}
if (minLon >= maxLon || minLat >= maxLat) {
return reply.code(400).send({ ok: false, error: "Invalid bbox bounds" });
}
if (zoomLevel < 0 || zoomLevel > 22) {
return reply.code(400).send({ ok: false, error: "Zoom level must be between 0 and 22" });
}
const cellSize = Math.max(0.01, 180 / Math.pow(2, zoomLevel));
// Source: band_locations (un point par localisation géocodée) JOIN bands.
const where = [
"bl.geom IS NOT NULL",
"bl.geocode_status IN ('done','country_only')",
];
const vals = [];
let i = 1;
where.push(`bl.geom && ST_MakeEnvelope($${i}, $${i+1}, $${i+2}, $${i+3}, 4326)`);
vals.push(minLon, minLat, maxLon, maxLat);
i += 4;
if (countries) {
const cs = String(countries).split(",").map(s => s.trim().toUpperCase()).filter(Boolean);
if (cs.length > 100) {
return reply.code(400).send({ ok: false, error: "Max 100 pays simultanés" });
}
if (cs.length) {
where.push(`b.country = ANY($${i}::text[])`);
vals.push(cs);
i++;
}
}
if (status) {
const st = String(status).split(",").map(s => s.trim()).filter(Boolean);
if (st.length > 50) {
return reply.code(400).send({ ok: false, error: "Max 50 statuts simultanés" });
}
if (st.length) {
where.push(`COALESCE(NULLIF(trim(b.status), ''), 'Unknown') = ANY($${i}::text[])`);
vals.push(st);
i++;
}
}
if (genre) {
try {
const genreQuery = sanitizeSearchString(genre, 'Genre');
where.push(`b.genre ILIKE $${i}`);
vals.push(`%${genreQuery}%`);
i++;
} catch (err) {
return reply.code(400).send({ ok: false, error: err.message });
}
}
if (year_min) {
const yearMin = Number(year_min);
if (!Number.isFinite(yearMin) || yearMin < 1800 || yearMin > 2100) {
return reply.code(400).send({ ok: false, error: 'year_min invalide' });
}
where.push(`b.formed_year >= $${i}`);
vals.push(yearMin);
i++;
}
if (year_max) {
const yearMax = Number(year_max);
if (!Number.isFinite(yearMax) || yearMax < 1800 || yearMax > 2100) {
return reply.code(400).send({ ok: false, error: 'year_max invalide' });
}
where.push(`b.formed_year <= $${i}`);
vals.push(yearMax);
i++;
}
const whereSql = where.join(" AND ");
if (zoomLevel >= 12) {
const sql = `
SELECT
b.ma_id,
b.name,
b.country,
COALESCE(NULLIF(trim(b.status), ''), 'Unknown') as status,
b.genre,
b.location_text,
b.formed_year,
bl.lat,
bl.lon,
bl.location_raw,
bl.step_order,
bl.step_label,
bl.is_country_only
FROM band_locations bl
JOIN bands b ON b.ma_id = bl.ma_id
WHERE ${whereSql}
ORDER BY bl.ma_id ASC, bl.step_order ASC
LIMIT 2000;
`;
const r = await p.query(sql, vals);
return {
ok: true,
type: "bands",
items: r.rows,
count: r.rows.length
};
}
const sql = `
WITH grid_cells AS (
SELECT
floor(ST_X(bl.geom) / $${i})::int as cell_x,
floor(ST_Y(bl.geom) / $${i})::int as cell_y,
b.ma_id,
b.name,
b.country,
COALESCE(NULLIF(trim(b.status), ''), 'Unknown') as status,
b.genre,
b.location_text,
b.formed_year,
bl.lat,
bl.lon,
bl.location_raw,
bl.step_label
FROM band_locations bl
JOIN bands b ON b.ma_id = bl.ma_id
WHERE ${whereSql}
),
clusters AS (
SELECT
cell_x,
cell_y,
count(*)::int as band_count,
avg(lat)::float8 as center_lat,
avg(lon)::float8 as center_lon,
array_agg(json_build_object(
'ma_id', ma_id,
'name', name,
'country', country,
'status', status,
'genre', genre,
'location_text', location_text,
'formed_year', formed_year,
'lat', lat,
'lon', lon,
'location_raw', location_raw,
'step_label', step_label
) ORDER BY name) as bands
FROM grid_cells
GROUP BY cell_x, cell_y
)
SELECT
center_lat as lat,
center_lon as lon,
band_count as count,
CASE
WHEN band_count <= 5 THEN bands
ELSE bands[1:5]
END as sample_bands
FROM clusters
ORDER BY band_count DESC
LIMIT 1000;
`;
vals.push(cellSize);
const r = await p.query(sql, vals);
return {
ok: true,
type: "clusters",
items: r.rows,
total_clusters: r.rows.length,
cell_size: cellSize,
zoom: zoomLevel
};
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des clusters' });
}
});
fastify.get("/api/bands", async (req, reply) => {
try {
const p = requirePool();
const {
countries,
geocoded,
q,
status,
only_black,
limit,
offset,
} = /** @type {Record<string, string|undefined>} */ (req.query || {});
let lim, off;
try {
({ limit: lim, offset: off } = parseLimitOffset(limit, offset, {
defaultLimit: 1000,
maxLimit: 150000,
}));
} catch (err) {
if (err instanceof ValidationError) {
return reply.code(400).send({ ok: false, error: err.message });
}
throw err;
}
if (lim > 10000) {
fastify.log.warn(`Large query: ${lim} rows requested from ${req.ip}`);
}
const where = [];
const vals = [];
let i = 1;
if (countries) {
const cs = String(countries).split(",").map(s => s.trim().toUpperCase()).filter(Boolean);
if (cs.length > 100) {
return reply.code(400).send({ ok: false, error: "Max 100 pays simultanés" });
}
if (cs.length) {
where.push(`country = ANY($${i}::text[])`);
vals.push(cs);
i++;
}
}
if (geocoded === "1") where.push(`geom IS NOT NULL`);
if (geocoded === "0") where.push(`geom IS NULL`);
if (status) {
const st = String(status).split(",").map(s => s.trim()).filter(Boolean);
if (st.length > 50) {
return reply.code(400).send({ ok: false, error: "Max 50 statuts simultanés" });
}
if (st.length) {
where.push(`COALESCE(NULLIF(trim(status), ''), 'Unknown') = ANY($${i}::text[])`);
vals.push(st);
i++;
}
}
if (only_black === "1") {
where.push(`genre ILIKE '%black%'`);
}
if (q) {
try {
const query = sanitizeSearchString(q, 'Recherche');
const qq = `%${query}%`;
where.push(`(name ILIKE $${i} OR genre ILIKE $${i} OR location_text ILIKE $${i} OR COALESCE(status,'') ILIKE $${i})`);
vals.push(qq);
i++;
} catch (err) {
return reply.code(400).send({ ok: false, error: err.message });
}
}
const whereSql = where.length ? `WHERE ${where.join(" AND ")}` : "";
const sql = `
SELECT
ma_id,
name,
country,
COALESCE(NULLIF(trim(status), ''), 'Unknown') as status,
genre,
location_text,
enriched,
formed_year,
themes,
lat,
lon
FROM bands
${whereSql}
ORDER BY ma_id ASC
LIMIT $${i} OFFSET $${i+1};
`;
vals.push(lim, off);
const r = await p.query(sql, vals);
return {
ok: true,
items: r.rows,
limit: lim,
offset: off
};
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la recherche' });
}
});
fastify.get("/api/facets", async (req, reply) => {
try {
const p = requirePool();
const [statusRes, countryRes, genreRes, yearRes] = await Promise.all([
p.query(`
SELECT
COALESCE(NULLIF(trim(status), ''), 'Unknown') as value,
count(*)::int as count
FROM bands
WHERE geom IS NOT NULL
GROUP BY 1
ORDER BY count DESC
`),
p.query(`
SELECT
COALESCE(country, '??') as value,
count(*)::int as count
FROM bands
WHERE geom IS NOT NULL
GROUP BY 1
ORDER BY count DESC
`),
p.query(`
SELECT
genre as value,
count(*)::int as count
FROM bands
WHERE geom IS NOT NULL
AND genre IS NOT NULL
AND genre != ''
GROUP BY 1
ORDER BY count DESC
LIMIT 500
`),
p.query(`
SELECT
MIN(formed_year)::int as min_year,
MAX(formed_year)::int as max_year
FROM bands
WHERE geom IS NOT NULL
AND formed_year >= 1900
AND formed_year <= extract(year from now())
`)
]);
return {
ok: true,
statuses: statusRes.rows,
countries: countryRes.rows,
genres: genreRes.rows,
year_range: yearRes.rows[0] || { min_year: null, max_year: null }
};
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des facettes' });
}
});
fastify.get("/api/band/:ma_id", async (req, reply) => {
try {
const p = requirePool();
const { ma_id } = /** @type {{ ma_id: string }} */ (req.params);
const id = Number(ma_id);
if (!Number.isFinite(id) || id < 0) {
return reply.code(400).send({ ok: false, error: "bad ma_id" });
}
const r = await p.query(
`SELECT
ma_id,
name,
country,
status,
genre,
location_text,
enriched,
data,
formed_year,
themes,
lat,
lon,
geocoded_at
FROM bands
WHERE ma_id = $1
LIMIT 1`,
[id]
);
if (!r.rows.length) {
return reply.code(404).send({ ok: false, error: "not found" });
}
return { ok: true, item: r.rows[0] };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération du groupe' });
}
});
fastify.register(async function(adminRoutes) {
await adminRoutes.register(rateLimit, {
max: adminRateLimitMax,
timeWindow: '1 minute',
keyGenerator: (req) => {
return authBearer(req) || req.ip;
}
});
adminRoutes.post("/admin/import", async (req, reply) => {
if (!requireAdmin(req, reply)) return;
try {
const body = /** @type {{ bands?: unknown[] }} */ (req.body);
if (!body || typeof body !== "object") {
return reply.code(400).send({ ok: false, error: "invalid body/json" });
}
const bands = Array.isArray(body.bands) ? body.bands : null;
if (!bands) {
return reply.code(400).send({ ok: false, error: "missing bands[]" });
}
if (bands.length > 1000) {
return reply.code(400).send({
ok: false,
error: "Max 1000 bands par import"
});
}
const p = requirePool();
let upserted = 0;
for (const raw of bands) {
if (!raw || typeof raw !== "object") continue;
const b = /** @type {Record<string, any>} */ (raw);
const ma_id = Number(b.ma_id);
if (!Number.isFinite(ma_id)) continue;
const name = b.name ?? null;
const country = b.country ?? null;
const status = b.status ?? null;
const genre = b.genre ?? null;
const location_text = b.location_text ?? b.location ?? null;
const hasData = b.data && typeof b.data === "object";
const data = hasData ? b.data : null;
const enriched = hasData ? true : (b.enriched ?? null);
await p.query(
`
INSERT INTO bands (ma_id, name, country, status, genre, location_text, data, enriched)
VALUES ($1,$2,$3,$4,$5,$6,$7, COALESCE($8,false))
ON CONFLICT (ma_id) DO UPDATE SET
name = COALESCE(EXCLUDED.name, bands.name),
country = COALESCE(EXCLUDED.country, bands.country),
status = COALESCE(EXCLUDED.status, bands.status),
genre = COALESCE(EXCLUDED.genre, bands.genre),
location_text = COALESCE(EXCLUDED.location_text, bands.location_text),
data = COALESCE(EXCLUDED.data, bands.data),
enriched = COALESCE(EXCLUDED.enriched, bands.enriched)
`,
[ma_id, name, country, status, genre, location_text, data, enriched]
);
upserted++;
}
return reply.send({ ok: true, upserted });
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de l\'import' });
}
});
adminRoutes.get("/admin/enrich/next", async (req, reply) => {
if (!requireAdmin(req, reply)) return;
try {
const p = requirePool();
const query = /** @type {Record<string, string|undefined>} */ (req.query || {});
const limitRaw = Number(query.limit ?? 50);
const limit = Math.max(1, Math.min(Number.isFinite(limitRaw) ? limitRaw : 50, 500));
const country = (query.country ?? "").toString().trim().toUpperCase();
const params = [];
let where = "WHERE (data->'band_page') IS NULL";
if (country) {
params.push(country);
where += ` AND country = $${params.length}`;
}
params.push(limit);
const sql = `
SELECT
ma_id,
(data->>'url') AS url
FROM bands
${where}
ORDER BY ma_id ASC
LIMIT $${params.length}
`;
const r = await p.query(sql, params);
const items = (r.rows || []).filter((x) => x.url);
return { ok: true, count: items.length, items };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des groupes à enrichir' });
}
});
});
// ------------------------------------------------------------------
// Auth dashboard admin (session cookie, distincte du token importToken)
// ------------------------------------------------------------------
fastify.register(async function (authRoutes) {
await authRoutes.register(rateLimit, {
max: authRateLimitMax,
timeWindow: "1 minute",
keyGenerator: (req) => req.ip,
});
authRoutes.post("/admin/auth/login", async (req, reply) => {
try {
const p = requirePool();
const { username, password } = /** @type {{ username?: unknown, password?: unknown }} */ (req.body || {});
if (typeof username !== "string" || typeof password !== "string" || !username || !password) {
return reply.code(400).send({ ok: false, error: "username et password requis" });
}
const uname = username.trim().slice(0, 100);
const ip = req.ip;
if (await isLockedOut(p, uname, ip)) {
return reply.code(429).send({ ok: false, error: "Trop de tentatives, réessayez dans quelques minutes" });
}
const valid = await verifyPassword(p, uname, password);
await recordLoginAttempt(p, uname, ip, valid);
if (!valid) {
return reply.code(401).send({ ok: false, error: "Identifiants invalides" });
}
await markLoginSuccess(p, uname);
const token = signAdminSession(uname);
reply.setCookie(ADMIN_COOKIE_NAME, token, {
httpOnly: true,
secure: true,
sameSite: "strict",
path: "/",
maxAge: 12 * 3600,
});
return { ok: true, username: uname };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: "Erreur de connexion" });
}
});
authRoutes.post("/admin/auth/logout", async (req, reply) => {
reply.clearCookie(ADMIN_COOKIE_NAME, { path: "/" });
return { ok: true };
});
authRoutes.get("/admin/auth/me", async (req, reply) => {
const username = requireAdminSession(req, reply);
if (!username) return;
return { ok: true, username };
});
});
await fastify.register(adminApiRoutes, { pool });
await fastify.ready();
return fastify;
}

View file

@ -1,811 +1,25 @@
import Fastify from "fastify"; /**
* Point d'entrée du process. Toute la logique vit dans app.js pour rester
* testable ; ce fichier ne fait qu'ouvrir la connexion DB et écouter.
*/
import pg from "pg"; import pg from "pg";
import rateLimit from "@fastify/rate-limit"; import { buildServer } from "./app.js";
import helmet from "@fastify/helmet";
import cookie from "@fastify/cookie";
import { timingSafeEqual } from "crypto";
import {
ADMIN_COOKIE_NAME,
seedAdminUser,
signAdminSession,
isLockedOut,
recordLoginAttempt,
verifyPassword,
markLoginSuccess,
requireAdminSession,
} from "./adminAuth.js";
import adminApiRoutes from "./adminRoutes.js";
const { Pool } = pg; const { Pool } = pg;
const fastify = Fastify({
logger: true,
bodyLimit: 10485760, // 10 MB max pour éviter DoS mémoire
trustProxy: true // derrière Traefik (+ nginx pour /admin/*) : lit X-Forwarded-For
});
// Headers de sécurité
await fastify.register(helmet, {
contentSecurityPolicy: false,
crossOriginEmbedderPolicy: false
});
await fastify.register(cookie);
// CORS middleware
fastify.addHook('onRequest', async (request, reply) => {
const origin = request.headers.origin;
const allowedOrigins = process.env.CORS_ORIGINS
? process.env.CORS_ORIGINS.split(',').map(s => s.trim())
: ['https://metalfrom.eu', 'https://www.metalfrom.eu'];
if (allowedOrigins.includes(origin)) {
reply.header('Access-Control-Allow-Origin', origin);
}
reply.header('Access-Control-Allow-Methods', 'GET, POST, PUT, PATCH, DELETE, OPTIONS');
reply.header('Access-Control-Allow-Headers', 'Content-Type, Authorization');
reply.header('Access-Control-Allow-Credentials', 'true');
if (request.method === 'OPTIONS') {
reply.status(204).send();
}
});
// Rate limiting global
await fastify.register(rateLimit, {
max: 1000,
timeWindow: '1 minute',
cache: 10000
});
// Gestion d'erreurs globale
fastify.setErrorHandler((error, request, reply) => {
fastify.log.error(error);
reply.status(500).send({
ok: false,
error: 'Une erreur est survenue'
});
});
const PORT = Number(process.env.PORT || 3000); const PORT = Number(process.env.PORT || 3000);
const DATABASE_URL = process.env.DATABASE_URL; const DATABASE_URL = process.env.DATABASE_URL;
const BM_IMPORT_TOKEN = (process.env.BM_IMPORT_TOKEN || "").trim();
let pool = null; const pool = DATABASE_URL
if (DATABASE_URL) { ? new Pool({ connectionString: DATABASE_URL, statement_timeout: 60000 })
pool = new Pool({ : null;
connectionString: DATABASE_URL,
statement_timeout: 60000
});
seedAdminUser(pool).catch((err) => fastify.log.error({ err }, "[admin] seed failed"));
}
function requirePool() { const fastify = await buildServer({ pool, logger: true });
if (!pool) throw new Error("DATABASE_URL not set");
return pool;
}
function authBearer(req) { // `listen()` renvoie une promesse : sans catch, un port déjà pris produisait un
const h = req.headers.authorization || ""; // unhandled rejection et un process zombie au lieu d'un exit code non nul (que
const m = h.match(/^Bearer\s+(.+)$/i); // Docker/Coolify utilisent pour redémarrer le conteneur).
return m ? m[1] : null; fastify.listen({ port: PORT, host: "0.0.0.0" }).catch((err) => {
} fastify.log.error(err, "[api] listen failed");
process.exit(1);
function requireAdmin(req, reply) {
const provided = authBearer(req);
if (!BM_IMPORT_TOKEN || !provided || provided.length !== BM_IMPORT_TOKEN.length) {
reply.code(401).send({ ok: false, error: "unauthorized" });
return false;
}
const valid = timingSafeEqual(
Buffer.from(provided),
Buffer.from(BM_IMPORT_TOKEN)
);
if (!valid) {
reply.code(401).send({ ok: false, error: "unauthorized" });
return false;
}
return true;
}
function sanitizeSearchString(str, fieldName = 'champ') {
const trimmed = String(str).trim();
if (trimmed.length < 2 || trimmed.length > 100) {
throw new Error(`${fieldName} doit faire entre 2 et 100 caractères`);
}
if (/[\x00-\x08\x0B-\x0C\x0E-\x1F\x7F]/.test(trimmed)) {
throw new Error(`${fieldName} contient des caractères invalides`);
}
if (/%%%|_{10,}|%{10,}/.test(trimmed)) {
throw new Error(`${fieldName} contient des patterns invalides`);
}
return trimmed;
}
fastify.get("/", async () => {
return { ok: true, service: "BM API" };
}); });
fastify.get("/api/health", async () => ({ ok: true }));
fastify.get("/api/db", async (req, reply) => {
try {
const p = requirePool();
const r = await p.query("SELECT now() as now, current_database() as db");
return { ok: true, ...r.rows[0] };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur de connexion à la base de données' });
}
});
fastify.get("/api/stats", async (req, reply) => {
try {
const p = requirePool();
const q = `
SELECT
count(*)::int as total,
count(*) FILTER (WHERE geom IS NOT NULL)::int as geocoded,
count(*) FILTER (WHERE geom IS NULL)::int as no_location,
count(*) FILTER (WHERE enriched = true)::int as enriched
FROM bands;
`;
const r = await p.query(q);
return { ok: true, ...r.rows[0] };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des stats' });
}
});
fastify.get("/api/countries", async (req, reply) => {
try {
const p = requirePool();
const q = `
SELECT
country,
count(*)::int as total,
count(*) FILTER (WHERE geom IS NOT NULL)::int as geocoded,
count(*) FILTER (WHERE enriched = true)::int as enriched
FROM bands
GROUP BY country
ORDER BY total DESC;
`;
const r = await p.query(q);
return { ok: true, items: r.rows };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des pays' });
}
});
fastify.get("/api/statuses", async (req, reply) => {
try {
const p = requirePool();
const q = `
SELECT
COALESCE(NULLIF(trim(status), ''), 'Unknown') as status,
count(*)::int as total,
count(*) FILTER (WHERE geom IS NOT NULL)::int as geocoded
FROM bands
GROUP BY 1
ORDER BY total DESC;
`;
const r = await p.query(q);
return { ok: true, items: r.rows };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des statuts' });
}
});
fastify.get("/api/clusters", async (req, reply) => {
try {
const p = requirePool();
const {
bbox,
zoom,
countries,
status,
genre,
year_min,
year_max,
} = req.query || {};
if (!bbox || !zoom) {
return reply.code(400).send({ ok: false, error: "bbox and zoom are required" });
}
const bboxParts = String(bbox).split(",").map(Number);
if (bboxParts.length !== 4) {
return reply.code(400).send({ ok: false, error: "bbox must have 4 values" });
}
const [minLon, minLat, maxLon, maxLat] = bboxParts;
const zoomLevel = Number(zoom);
if ([minLon, minLat, maxLon, maxLat, zoomLevel].some(v => !Number.isFinite(v))) {
return reply.code(400).send({ ok: false, error: "Invalid bbox or zoom values" });
}
if (minLat < -90 || maxLat > 90 || minLon < -180 || maxLon > 180) {
return reply.code(400).send({ ok: false, error: "Coordinates out of range" });
}
if (minLon >= maxLon || minLat >= maxLat) {
return reply.code(400).send({ ok: false, error: "Invalid bbox bounds" });
}
if (zoomLevel < 0 || zoomLevel > 22) {
return reply.code(400).send({ ok: false, error: "Zoom level must be between 0 and 22" });
}
const cellSize = Math.max(0.01, 180 / Math.pow(2, zoomLevel));
// Source: band_locations (un point par localisation géocodée) JOIN bands.
const where = [
"bl.geom IS NOT NULL",
"bl.geocode_status IN ('done','country_only')",
];
const vals = [];
let i = 1;
where.push(`bl.geom && ST_MakeEnvelope($${i}, $${i+1}, $${i+2}, $${i+3}, 4326)`);
vals.push(minLon, minLat, maxLon, maxLat);
i += 4;
if (countries) {
const cs = String(countries).split(",").map(s => s.trim().toUpperCase()).filter(Boolean);
if (cs.length > 100) {
return reply.code(400).send({ ok: false, error: "Max 100 pays simultanés" });
}
if (cs.length) {
where.push(`b.country = ANY($${i}::text[])`);
vals.push(cs);
i++;
}
}
if (status) {
const st = String(status).split(",").map(s => s.trim()).filter(Boolean);
if (st.length > 50) {
return reply.code(400).send({ ok: false, error: "Max 50 statuts simultanés" });
}
if (st.length) {
where.push(`COALESCE(NULLIF(trim(b.status), ''), 'Unknown') = ANY($${i}::text[])`);
vals.push(st);
i++;
}
}
if (genre) {
try {
const genreQuery = sanitizeSearchString(genre, 'Genre');
where.push(`b.genre ILIKE $${i}`);
vals.push(`%${genreQuery}%`);
i++;
} catch (err) {
return reply.code(400).send({ ok: false, error: err.message });
}
}
if (year_min) {
const yearMin = Number(year_min);
if (!Number.isFinite(yearMin) || yearMin < 1800 || yearMin > 2100) {
return reply.code(400).send({ ok: false, error: 'year_min invalide' });
}
where.push(`b.formed_year >= $${i}`);
vals.push(yearMin);
i++;
}
if (year_max) {
const yearMax = Number(year_max);
if (!Number.isFinite(yearMax) || yearMax < 1800 || yearMax > 2100) {
return reply.code(400).send({ ok: false, error: 'year_max invalide' });
}
where.push(`b.formed_year <= $${i}`);
vals.push(yearMax);
i++;
}
const whereSql = where.join(" AND ");
if (zoomLevel >= 12) {
const sql = `
SELECT
b.ma_id,
b.name,
b.country,
COALESCE(NULLIF(trim(b.status), ''), 'Unknown') as status,
b.genre,
b.location_text,
b.formed_year,
bl.lat,
bl.lon,
bl.location_raw,
bl.step_order,
bl.step_label,
bl.is_country_only
FROM band_locations bl
JOIN bands b ON b.ma_id = bl.ma_id
WHERE ${whereSql}
ORDER BY bl.ma_id ASC, bl.step_order ASC
LIMIT 2000;
`;
const r = await p.query(sql, vals);
return {
ok: true,
type: "bands",
items: r.rows,
count: r.rows.length
};
}
const sql = `
WITH grid_cells AS (
SELECT
floor(ST_X(bl.geom) / $${i})::int as cell_x,
floor(ST_Y(bl.geom) / $${i})::int as cell_y,
b.ma_id,
b.name,
b.country,
COALESCE(NULLIF(trim(b.status), ''), 'Unknown') as status,
b.genre,
b.location_text,
b.formed_year,
bl.lat,
bl.lon,
bl.location_raw,
bl.step_label
FROM band_locations bl
JOIN bands b ON b.ma_id = bl.ma_id
WHERE ${whereSql}
),
clusters AS (
SELECT
cell_x,
cell_y,
count(*)::int as band_count,
avg(lat)::float8 as center_lat,
avg(lon)::float8 as center_lon,
array_agg(json_build_object(
'ma_id', ma_id,
'name', name,
'country', country,
'status', status,
'genre', genre,
'location_text', location_text,
'formed_year', formed_year,
'lat', lat,
'lon', lon,
'location_raw', location_raw,
'step_label', step_label
) ORDER BY name) as bands
FROM grid_cells
GROUP BY cell_x, cell_y
)
SELECT
center_lat as lat,
center_lon as lon,
band_count as count,
CASE
WHEN band_count <= 5 THEN bands
ELSE bands[1:5]
END as sample_bands
FROM clusters
ORDER BY band_count DESC
LIMIT 1000;
`;
vals.push(cellSize);
const r = await p.query(sql, vals);
return {
ok: true,
type: "clusters",
items: r.rows,
total_clusters: r.rows.length,
cell_size: cellSize,
zoom: zoomLevel
};
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des clusters' });
}
});
fastify.get("/api/bands", async (req, reply) => {
try {
const p = requirePool();
const {
countries,
geocoded,
q,
status,
only_black,
limit,
offset,
} = req.query || {};
const lim = Math.min(Number(limit || 1000), 150000);
const off = Math.max(Number(offset || 0), 0);
if (off > 1000000) {
return reply.code(400).send({
ok: false,
error: "Offset trop grand (max 1 million)"
});
}
if (lim > 10000) {
fastify.log.warn(`Large query: ${lim} rows requested from ${req.ip}`);
}
const where = [];
const vals = [];
let i = 1;
if (countries) {
const cs = String(countries).split(",").map(s => s.trim().toUpperCase()).filter(Boolean);
if (cs.length > 100) {
return reply.code(400).send({ ok: false, error: "Max 100 pays simultanés" });
}
if (cs.length) {
where.push(`country = ANY($${i}::text[])`);
vals.push(cs);
i++;
}
}
if (geocoded === "1") where.push(`geom IS NOT NULL`);
if (geocoded === "0") where.push(`geom IS NULL`);
if (status) {
const st = String(status).split(",").map(s => s.trim()).filter(Boolean);
if (st.length > 50) {
return reply.code(400).send({ ok: false, error: "Max 50 statuts simultanés" });
}
if (st.length) {
where.push(`COALESCE(NULLIF(trim(status), ''), 'Unknown') = ANY($${i}::text[])`);
vals.push(st);
i++;
}
}
if (only_black === "1") {
where.push(`genre ILIKE '%black%'`);
}
if (q) {
try {
const query = sanitizeSearchString(q, 'Recherche');
const qq = `%${query}%`;
where.push(`(name ILIKE $${i} OR genre ILIKE $${i} OR location_text ILIKE $${i} OR COALESCE(status,'') ILIKE $${i})`);
vals.push(qq);
i++;
} catch (err) {
return reply.code(400).send({ ok: false, error: err.message });
}
}
const whereSql = where.length ? `WHERE ${where.join(" AND ")}` : "";
const sql = `
SELECT
ma_id,
name,
country,
COALESCE(NULLIF(trim(status), ''), 'Unknown') as status,
genre,
location_text,
enriched,
formed_year,
themes,
lat,
lon
FROM bands
${whereSql}
ORDER BY ma_id ASC
LIMIT $${i} OFFSET $${i+1};
`;
vals.push(lim, off);
const r = await p.query(sql, vals);
return {
ok: true,
items: r.rows,
limit: lim,
offset: off
};
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la recherche' });
}
});
fastify.get("/api/facets", async (req, reply) => {
try {
const p = requirePool();
const [statusRes, countryRes, genreRes, yearRes] = await Promise.all([
p.query(`
SELECT
COALESCE(NULLIF(trim(status), ''), 'Unknown') as value,
count(*)::int as count
FROM bands
WHERE geom IS NOT NULL
GROUP BY 1
ORDER BY count DESC
`),
p.query(`
SELECT
COALESCE(country, '??') as value,
count(*)::int as count
FROM bands
WHERE geom IS NOT NULL
GROUP BY 1
ORDER BY count DESC
`),
p.query(`
SELECT
genre as value,
count(*)::int as count
FROM bands
WHERE geom IS NOT NULL
AND genre IS NOT NULL
AND genre != ''
GROUP BY 1
ORDER BY count DESC
LIMIT 500
`),
p.query(`
SELECT
MIN(formed_year)::int as min_year,
MAX(formed_year)::int as max_year
FROM bands
WHERE geom IS NOT NULL
AND formed_year >= 1900
AND formed_year <= extract(year from now())
`)
]);
return {
ok: true,
statuses: statusRes.rows,
countries: countryRes.rows,
genres: genreRes.rows,
year_range: yearRes.rows[0] || { min_year: null, max_year: null }
};
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des facettes' });
}
});
fastify.get("/api/band/:ma_id", async (req, reply) => {
try {
const p = requirePool();
const id = Number(req.params.ma_id);
if (!Number.isFinite(id) || id < 0) {
return reply.code(400).send({ ok: false, error: "bad ma_id" });
}
const r = await p.query(
`SELECT
ma_id,
name,
country,
status,
genre,
location_text,
enriched,
data,
formed_year,
themes,
lat,
lon,
geocoded_at
FROM bands
WHERE ma_id = $1
LIMIT 1`,
[id]
);
if (!r.rows.length) {
return reply.code(404).send({ ok: false, error: "not found" });
}
return { ok: true, item: r.rows[0] };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération du groupe' });
}
});
fastify.register(async function(adminRoutes) {
await adminRoutes.register(rateLimit, {
max: 10,
timeWindow: '1 minute',
keyGenerator: (req) => {
return authBearer(req) || req.ip;
}
});
adminRoutes.post("/admin/import", async (req, reply) => {
if (!requireAdmin(req, reply)) return;
try {
const body = req.body;
if (!body || typeof body !== "object") {
return reply.code(400).send({ ok: false, error: "invalid body/json" });
}
const bands = Array.isArray(body.bands) ? body.bands : null;
if (!bands) {
return reply.code(400).send({ ok: false, error: "missing bands[]" });
}
if (bands.length > 1000) {
return reply.code(400).send({
ok: false,
error: "Max 1000 bands par import"
});
}
const p = requirePool();
let upserted = 0;
for (const b of bands) {
if (!b || typeof b !== "object") continue;
const ma_id = Number(b.ma_id);
if (!Number.isFinite(ma_id)) continue;
const name = b.name ?? null;
const country = b.country ?? null;
const status = b.status ?? null;
const genre = b.genre ?? null;
const location_text = b.location_text ?? b.location ?? null;
const hasData = b.data && typeof b.data === "object";
const data = hasData ? b.data : null;
const enriched = hasData ? true : (b.enriched ?? null);
await p.query(
`
INSERT INTO bands (ma_id, name, country, status, genre, location_text, data, enriched)
VALUES ($1,$2,$3,$4,$5,$6,$7, COALESCE($8,false))
ON CONFLICT (ma_id) DO UPDATE SET
name = COALESCE(EXCLUDED.name, bands.name),
country = COALESCE(EXCLUDED.country, bands.country),
status = COALESCE(EXCLUDED.status, bands.status),
genre = COALESCE(EXCLUDED.genre, bands.genre),
location_text = COALESCE(EXCLUDED.location_text, bands.location_text),
data = COALESCE(EXCLUDED.data, bands.data),
enriched = COALESCE(EXCLUDED.enriched, bands.enriched)
`,
[ma_id, name, country, status, genre, location_text, data, enriched]
);
upserted++;
}
return reply.send({ ok: true, upserted });
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de l\'import' });
}
});
adminRoutes.get("/admin/enrich/next", async (req, reply) => {
if (!requireAdmin(req, reply)) return;
try {
const p = requirePool();
const limitRaw = Number(req.query?.limit ?? 50);
const limit = Math.max(1, Math.min(Number.isFinite(limitRaw) ? limitRaw : 50, 500));
const country = (req.query?.country ?? "").toString().trim().toUpperCase();
const params = [];
let where = "WHERE (data->'band_page') IS NULL";
if (country) {
params.push(country);
where += ` AND country = $${params.length}`;
}
params.push(limit);
const sql = `
SELECT
ma_id,
(data->>'url') AS url
FROM bands
${where}
ORDER BY ma_id ASC
LIMIT $${params.length}
`;
const r = await p.query(sql, params);
const items = (r.rows || []).filter((x) => x.url);
return { ok: true, count: items.length, items };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: 'Erreur lors de la récupération des groupes à enrichir' });
}
});
});
// ------------------------------------------------------------------
// Auth dashboard admin (session cookie, distincte du token BM_IMPORT_TOKEN)
// ------------------------------------------------------------------
fastify.register(async function (authRoutes) {
await authRoutes.register(rateLimit, {
max: 10,
timeWindow: "1 minute",
keyGenerator: (req) => req.ip,
});
authRoutes.post("/admin/auth/login", async (req, reply) => {
try {
const p = requirePool();
const { username, password } = req.body || {};
if (typeof username !== "string" || typeof password !== "string" || !username || !password) {
return reply.code(400).send({ ok: false, error: "username et password requis" });
}
const uname = username.trim().slice(0, 100);
const ip = req.ip;
if (await isLockedOut(p, uname, ip)) {
return reply.code(429).send({ ok: false, error: "Trop de tentatives, réessayez dans quelques minutes" });
}
const valid = await verifyPassword(p, uname, password);
await recordLoginAttempt(p, uname, ip, valid);
if (!valid) {
return reply.code(401).send({ ok: false, error: "Identifiants invalides" });
}
await markLoginSuccess(p, uname);
const token = signAdminSession(uname);
reply.setCookie(ADMIN_COOKIE_NAME, token, {
httpOnly: true,
secure: true,
sameSite: "strict",
path: "/",
maxAge: 12 * 3600,
});
return { ok: true, username: uname };
} catch (err) {
fastify.log.error(err);
return reply.code(500).send({ ok: false, error: "Erreur de connexion" });
}
});
authRoutes.post("/admin/auth/logout", async (req, reply) => {
reply.clearCookie(ADMIN_COOKIE_NAME, { path: "/" });
return { ok: true };
});
authRoutes.get("/admin/auth/me", async (req, reply) => {
const username = requireAdminSession(req, reply);
if (!username) return;
return { ok: true, username };
});
});
await fastify.register(adminApiRoutes, { pool });
fastify.listen({ port: PORT, host: "0.0.0.0" });

161
apps/api/src/validate.js Normal file
View file

@ -0,0 +1,161 @@
/**
* Helpers de validation purs (aucune dépendance à Fastify / pg).
*
* Extraits de server.js / adminRoutes.js pour être testables unitairement sans
* démarrer de serveur ni de base de données.
*
* Convention : chaque helper renvoie soit la valeur normalisée, soit lève une
* ValidationError dont le `message` est directement affichable à l'utilisateur.
*/
export class ValidationError extends Error {
constructor(message) {
super(message);
this.name = "ValidationError";
}
}
/**
* Nettoie une chaîne de recherche destinée à un ILIKE.
* Rejette les caractères de contrôle et les patterns pathologiques qui font
* exploser le planner Postgres (backtracking sur `%%%`, `____________`, ).
*/
export function sanitizeSearchString(str, fieldName = "champ") {
const trimmed = String(str).trim();
if (trimmed.length < 2 || trimmed.length > 100) {
throw new ValidationError(`${fieldName} doit faire entre 2 et 100 caractères`);
}
// eslint-disable-next-line no-control-regex
if (/[\x00-\x08\x0B-\x0C\x0E-\x1F\x7F]/.test(trimmed)) {
throw new ValidationError(`${fieldName} contient des caractères invalides`);
}
if (/%%%|_{10,}|%{10,}/.test(trimmed)) {
throw new ValidationError(`${fieldName} contient des patterns invalides`);
}
return trimmed;
}
/**
* Parse et valide un `limit`/`offset` de liste publique.
*
* Corrige un bug de l'implémentation initiale : `Number("abc")` donnait NaN,
* que `Math.min` propageait jusqu'au `LIMIT $n` de la requête, provoquant une
* erreur pg et un 500 au lieu d'un 400. On retombe désormais sur la valeur par
* défaut pour une entrée non numérique.
*/
export function parseLimitOffset(rawLimit, rawOffset, { defaultLimit = 1000, maxLimit = 10000 } = {}) {
const limitNum = rawLimit === undefined || rawLimit === null || rawLimit === "" ? defaultLimit : Number(rawLimit);
if (!Number.isFinite(limitNum) || limitNum < 1) {
throw new ValidationError("limit invalide");
}
const offsetNum = rawOffset === undefined || rawOffset === null || rawOffset === "" ? 0 : Number(rawOffset);
if (!Number.isFinite(offsetNum) || offsetNum < 0) {
throw new ValidationError("offset invalide");
}
if (offsetNum > 1_000_000) {
throw new ValidationError("Offset trop grand (max 1 million)");
}
return {
limit: Math.min(Math.floor(limitNum), maxLimit),
offset: Math.floor(offsetNum),
};
}
/** Pagination admin : page/pageSize bornés, offset dérivé. */
export function pagination(query = {}) {
const page = Math.max(1, Number(query.page) || 1);
const pageSize = Math.min(200, Math.max(1, Number(query.pageSize) || 50));
return { page, pageSize, offset: (page - 1) * pageSize };
}
/**
* Parse "minLon,minLat,maxLon,maxLat" et valide la cohérence géographique.
*/
export function parseBbox(raw) {
if (!raw) throw new ValidationError("bbox and zoom are required");
const parts = String(raw).split(",").map(Number);
if (parts.length !== 4) throw new ValidationError("bbox must have 4 values");
const [minLon, minLat, maxLon, maxLat] = parts;
if (parts.some((v) => !Number.isFinite(v))) {
throw new ValidationError("Invalid bbox or zoom values");
}
if (minLat < -90 || maxLat > 90 || minLon < -180 || maxLon > 180) {
throw new ValidationError("Coordinates out of range");
}
if (minLon >= maxLon || minLat >= maxLat) {
throw new ValidationError("Invalid bbox bounds");
}
return { minLon, minLat, maxLon, maxLat };
}
export function parseZoom(raw) {
const zoom = Number(raw);
if (!Number.isFinite(zoom)) throw new ValidationError("Invalid bbox or zoom values");
if (zoom < 0 || zoom > 22) throw new ValidationError("Zoom level must be between 0 and 22");
return zoom;
}
/** Taille de cellule de la grille de clustering, en degrés. */
export function cellSizeForZoom(zoom) {
return Math.max(0.01, 180 / Math.pow(2, zoom));
}
/**
* Découpe une liste CSV (pays, statuts) en tableau borné.
* `transform` permet de normaliser chaque élément (ex: toUpperCase pour les pays).
*
* @param {unknown} raw
* @param {{ max?: number, label?: string, transform?: (s: string) => string }} [opts]
* @returns {string[]}
*/
export function parseCsvList(raw, { max, label, transform = (s) => s } = {}) {
const items = String(raw)
.split(",")
.map((s) => transform(s.trim()))
.filter(Boolean);
if (max && items.length > max) {
throw new ValidationError(`Max ${max} ${label} simultanés`);
}
return items;
}
/** Valide une année de formation. Renvoie null si l'entrée est vide. */
export function parseYear(raw, fieldName = "year") {
if (raw === undefined || raw === null || raw === "") return null;
const year = Number(raw);
if (!Number.isFinite(year) || year < 1800 || year > 2100) {
throw new ValidationError(`${fieldName} invalide`);
}
return year;
}
/** Valide une latitude. `null` autorisé (efface la valeur). */
export function parseLat(raw) {
if (raw === null || raw === undefined || raw === "") return null;
const v = Number(raw);
if (!Number.isFinite(v) || v < -90 || v > 90) throw new ValidationError("lat invalide");
return v;
}
/** Valide une longitude. `null` autorisé (efface la valeur). */
export function parseLon(raw) {
if (raw === null || raw === undefined || raw === "") return null;
const v = Number(raw);
if (!Number.isFinite(v) || v < -180 || v > 180) throw new ValidationError("lon invalide");
return v;
}
/** Valide un identifiant Metal Archives (BIGINT positif). */
export function parseMaId(raw) {
const id = Number(raw);
if (!Number.isInteger(id) || id < 0 || id > Number.MAX_SAFE_INTEGER) {
throw new ValidationError("bad ma_id");
}
return id;
}

View file

@ -0,0 +1,132 @@
import { describe, it, expect, vi } from "vitest";
process.env.ADMIN_JWT_SECRET = "secret-de-test-suffisamment-long-pour-passer-32";
const {
signAdminSession,
verifyAdminSession,
isLockedOut,
verifyPassword,
requireAdminSession,
ADMIN_COOKIE_NAME,
} = await import("../src/adminAuth.js");
const { makeFakePool, rows } = await import("./helpers/fakePool.js");
describe("sessions JWT", () => {
it("aller-retour signature / vérification", () => {
expect(verifyAdminSession(signAdminSession("nico"))).toBe("nico");
});
it("rejette un jeton signé avec un autre secret", async () => {
const jwt = (await import("jsonwebtoken")).default;
const forged = jwt.sign({ sub: "nico" }, "un-autre-secret-de-32-caracteres-ok");
expect(verifyAdminSession(forged)).toBeNull();
});
it("rejette un jeton `alg: none` (attaque classique)", async () => {
const header = Buffer.from(JSON.stringify({ alg: "none", typ: "JWT" })).toString("base64url");
const payload = Buffer.from(JSON.stringify({ sub: "nico" })).toString("base64url");
expect(verifyAdminSession(`${header}.${payload}.`)).toBeNull();
});
it("rejette un jeton expiré", async () => {
const jwt = (await import("jsonwebtoken")).default;
const expired = jwt.sign({ sub: "nico" }, process.env.ADMIN_JWT_SECRET, { expiresIn: -10 });
expect(verifyAdminSession(expired)).toBeNull();
});
it("rejette un jeton sans sub exploitable", async () => {
const jwt = (await import("jsonwebtoken")).default;
const noSub = jwt.sign({ sub: { admin: true } }, process.env.ADMIN_JWT_SECRET);
expect(verifyAdminSession(noSub)).toBeNull();
});
it("rejette une chaîne quelconque sans lever d'exception", () => {
expect(verifyAdminSession("")).toBeNull();
expect(verifyAdminSession("aaa.bbb.ccc")).toBeNull();
});
});
describe("requireAdminSession", () => {
function fakeReply() {
const reply = {
statusCode: null,
body: null,
code(c) { reply.statusCode = c; return reply; },
send(b) { reply.body = b; return reply; },
};
return reply;
}
it("renvoie le username et ne répond pas quand le cookie est valide", () => {
const reply = fakeReply();
const req = { cookies: { [ADMIN_COOKIE_NAME]: signAdminSession("nico") } };
expect(requireAdminSession(req, reply)).toBe("nico");
expect(reply.statusCode).toBeNull();
});
it("répond 401 et renvoie null sans cookie", () => {
const reply = fakeReply();
expect(requireAdminSession({ cookies: {} }, reply)).toBeNull();
expect(reply.statusCode).toBe(401);
});
it("ne plante pas quand `cookies` est absent", () => {
const reply = fakeReply();
expect(requireAdminSession({}, reply)).toBeNull();
expect(reply.statusCode).toBe(401);
});
});
describe("verifyPassword", () => {
it("renvoie false pour un utilisateur inexistant", async () => {
const bcrypt = (await import("bcryptjs")).default;
// Le DUMMY_HASH de production est à coût 12 (~330 ms par comparaison).
// On neutralise le calcul : ce test porte sur la valeur de retour, et le
// test suivant vérifie séparément que la comparaison a bien lieu.
const spy = vi.spyOn(bcrypt, "compare").mockResolvedValue(true);
const pool = makeFakePool([{ match: "FROM admin_users", result: rows() }]);
// Même si bcrypt dit « vrai », l'absence d'utilisateur doit l'emporter.
expect(await verifyPassword(pool, "inconnu", "peu-importe")).toBe(false);
spy.mockRestore();
});
// Anti-énumération : le hash factice doit être comparé même quand l'utilisateur
// n'existe pas, sinon le temps de réponse révèle les comptes valides.
it("compare quand même un hash pour un utilisateur inexistant", async () => {
const bcrypt = (await import("bcryptjs")).default;
const spy = vi.spyOn(bcrypt, "compare").mockResolvedValue(false);
const pool = makeFakePool([{ match: "FROM admin_users", result: rows() }]);
await verifyPassword(pool, "inconnu", "x");
expect(spy).toHaveBeenCalled();
// La comparaison doit porter sur un hash bcrypt valide, sinon le temps de
// réponse trahirait l'absence de compte.
expect(spy.mock.calls[0][1]).toMatch(/^\$2[aby]\$\d\d\$/);
spy.mockRestore();
});
it("renvoie true pour le bon mot de passe", async () => {
const bcrypt = (await import("bcryptjs")).default;
const hash = bcrypt.hashSync("bon-mot-de-passe", 4); // coût faible : test rapide
const pool = makeFakePool([{ match: "FROM admin_users", result: rows({ password_hash: hash }) }]);
expect(await verifyPassword(pool, "nico", "bon-mot-de-passe")).toBe(true);
expect(await verifyPassword(pool, "nico", "mauvais")).toBe(false);
});
});
describe("isLockedOut", () => {
it("verrouille à partir de 5 tentatives échouées", async () => {
const under = makeFakePool([{ match: "admin_login_attempts", result: rows({ n: 4 }) }]);
const at = makeFakePool([{ match: "admin_login_attempts", result: rows({ n: 5 }) }]);
expect(await isLockedOut(under, "nico", "1.2.3.4")).toBe(false);
expect(await isLockedOut(at, "nico", "1.2.3.4")).toBe(true);
});
it("compte les tentatives par username OU par IP, sur 15 minutes", async () => {
const pool = makeFakePool([{ match: "admin_login_attempts", result: rows({ n: 0 }) }]);
await isLockedOut(pool, "nico", "1.2.3.4");
const call = pool.find("admin_login_attempts");
expect(call.sql).toMatch(/username = \$2 OR ip = \$3/);
expect(call.values).toEqual([15, "nico", "1.2.3.4"]);
});
});

View file

@ -0,0 +1,388 @@
import { describe, it, expect, beforeAll, afterAll } from "vitest";
// Doit être défini avant l'import d'adminAuth (jwtSecret() lit process.env à l'appel,
// mais on fixe la valeur ici pour que tous les tests partagent le même secret).
process.env.ADMIN_JWT_SECRET = "secret-de-test-suffisamment-long-pour-passer-32";
const { signAdminSession, ADMIN_COOKIE_NAME } = await import("../src/adminAuth.js");
const { rows } = await import("./helpers/fakePool.js");
const { createAdminApp } = await import("./helpers/testApp.js");
let sessionCookie, app, pool;
beforeAll(async () => {
sessionCookie = `${ADMIN_COOKIE_NAME}=${signAdminSession("nico")}`;
({ app, pool } = await createAdminApp());
});
afterAll(async () => { await app.close(); });
/** Reprogramme le pool partagé et renvoie l'app (construite une seule fois). */
function buildApp(handlers) {
pool.reset(handlers ?? []);
return app;
}
/** Requête authentifiée par défaut. */
function auth(headers = {}) {
return { cookie: sessionCookie, ...headers };
}
describe("garde d'authentification", () => {
it("refuse toute route admin sans cookie de session", async () => {
const app = buildApp(([]));
for (const url of ["/admin/api/stats", "/admin/api/bands", "/admin/api/live", "/admin/api/activity"]) {
const res = await app.inject({ method: "GET", url });
expect(res.statusCode, url).toBe(401);
}
});
it("refuse un cookie de session forgé", async () => {
const app = buildApp(([]));
const res = await app.inject({
method: "GET",
url: "/admin/api/stats",
headers: { cookie: `${ADMIN_COOKIE_NAME}=pas.un.jwt` },
});
expect(res.statusCode).toBe(401);
});
it("ne touche pas la base quand la session est invalide", async () => {
const handlers = ([]);
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/stats" });
expect(pool.calls).toHaveLength(0);
});
it("accepte une session valide", async () => {
const handlers = ([{ match: "FROM bands", result: rows({ total: 0 }) }]);
const app = buildApp(handlers);
const res = await app.inject({ method: "GET", url: "/admin/api/stats", headers: auth() });
expect(res.statusCode).toBe(200);
});
});
describe("GET /admin/api/bands — construction de la requête", () => {
const listPool = () =>
([
{ match: "count(*)::int AS total", result: rows({ total: 3 }) },
{ match: "SELECT ma_id, name", result: rows({ ma_id: 1, name: "Mayhem" }) },
]);
it("n'accepte que les colonnes de tri de l'allowlist", async () => {
const handlers = listPool();
const app = buildApp(handlers);
await app.inject({
method: "GET",
url: "/admin/api/bands?sort=name;DROP TABLE bands--&dir=desc",
headers: auth(),
});
const data = pool.find("SELECT ma_id, name");
expect(data.sql).not.toContain("DROP TABLE");
expect(data.sql).toContain("ORDER BY ma_id DESC");
});
it("accepte une colonne de tri légitime", async () => {
const handlers = listPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/bands?sort=formed_year", headers: auth() });
expect(pool.find("SELECT ma_id, name").sql).toContain("ORDER BY formed_year ASC");
});
it("passe les filtres en paramètres liés, jamais en interpolation", async () => {
const handlers = listPool();
const app = buildApp(handlers);
await app.inject({
method: "GET",
url: "/admin/api/bands?q=' OR 1=1--&country=fr",
headers: auth(),
});
const data = pool.find("SELECT ma_id, name");
expect(data.sql).not.toContain("OR 1=1");
expect(data.values).toContain("%' OR 1=1--%");
expect(data.values).toContain("FR"); // pays normalisé en majuscules
});
it("ignore une recherche de moins de 2 caractères", async () => {
const handlers = listPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/bands?q=a", headers: auth() });
expect(pool.find("SELECT ma_id, name").sql).not.toContain("ILIKE");
});
it("le count et la page utilisent le même WHERE", async () => {
const handlers = listPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/bands?country=de&enriched=true", headers: auth() });
const count = pool.find("count(*)::int AS total");
const data = pool.find("SELECT ma_id, name");
expect(count.sql).toContain("country = $1");
expect(data.sql).toContain("country = $1");
// le count ne doit pas recevoir les paramètres de pagination
expect(count.values).toEqual(["DE"]);
expect(data.values).toEqual(["DE", 50, 0]);
});
it("borne pageSize à 200", async () => {
const handlers = listPool();
const app = buildApp(handlers);
const res = await app.inject({ method: "GET", url: "/admin/api/bands?pageSize=99999", headers: auth() });
expect(res.json().pageSize).toBe(200);
});
});
describe("PATCH /admin/api/bands/:ma_id", () => {
const patchPool = () =>
([
{ match: "SELECT * FROM bands WHERE ma_id", result: rows({ ma_id: 1, name: "Avant" }) },
{ match: "UPDATE bands SET", result: rows({ ma_id: 1, name: "Après" }) },
{ match: "INSERT INTO admin_audit_log", result: rows() },
]);
it("refuse un ma_id non numérique", async () => {
const app = buildApp(patchPool());
const res = await app.inject({
method: "PATCH",
url: "/admin/api/bands/abc",
headers: auth(),
payload: { name: "x" },
});
expect(res.statusCode).toBe(400);
});
it("refuse un corps sans champ modifiable", async () => {
const app = buildApp(patchPool());
const res = await app.inject({
method: "PATCH",
url: "/admin/api/bands/1",
headers: auth(),
payload: { ma_id: 999, enriched: true, crawled_at: "2020-01-01" },
});
expect(res.statusCode).toBe(400);
expect(res.json().error).toMatch(/Aucun champ/);
});
it("ignore les champs hors allowlist mais applique les autres", async () => {
const handlers = patchPool();
const app = buildApp(handlers);
await app.inject({
method: "PATCH",
url: "/admin/api/bands/1",
headers: auth(),
payload: { name: "Darkthrone", enriched: false, locked_fields: "{}" },
});
const upd = pool.find("UPDATE bands SET");
expect(upd.sql).toContain("name = $2");
expect(upd.sql).not.toContain("enriched =");
expect(upd.sql).not.toContain("locked_fields = $");
});
it.each([
["formed_year", { formed_year: 1799 }],
["formed_year", { formed_year: 2101 }],
["lat", { lat: 91 }],
["lon", { lon: -181 }],
])("rejette %s hors bornes", async (_field, payload) => {
const app = buildApp(patchPool());
const res = await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth(), payload });
expect(res.statusCode).toBe(400);
});
it("verrouille les champs édités", async () => {
const handlers = patchPool();
const app = buildApp(handlers);
await app.inject({
method: "PATCH",
url: "/admin/api/bands/1",
headers: auth(),
payload: { name: "X", country: "NO" },
});
const upd = pool.find("UPDATE bands SET");
expect(upd.sql).toContain("locked_fields = locked_fields ||");
expect(upd.values.at(-1)).toBe(JSON.stringify({ name: true, country: true }));
});
it("écrit une entrée d'audit", async () => {
const handlers = patchPool();
const app = buildApp(handlers);
await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth(), payload: { name: "X" } });
const audit = pool.find("INSERT INTO admin_audit_log");
expect(audit).toBeDefined();
expect(audit.values[0]).toBe("nico"); // l'utilisateur de la session, pas une valeur du corps
});
it("404 si le groupe n'existe pas", async () => {
const handlers = ([{ match: "SELECT * FROM bands WHERE ma_id", result: rows() }]);
const app = buildApp(handlers);
const res = await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth(), payload: { name: "X" } });
expect(res.statusCode).toBe(404);
});
});
describe("POST /admin/api/bands/:ma_id/resolve-conflict", () => {
const conflictPool = () =>
([
{
match: "SELECT * FROM bands WHERE ma_id",
result: rows({ ma_id: 1, crawler_pending: { name: "MA", formed_year: 1991 } }),
},
{ match: "UPDATE bands SET", result: rows({ ma_id: 1 }) },
{ match: "INSERT INTO admin_audit_log", result: rows() },
]);
// Régression sécurité : `field` était interpolé dans le SET sans allowlist,
// ce qui donnait une injection SQL à tout admin authentifié.
it("refuse un nom de champ hors allowlist", async () => {
const handlers = conflictPool();
const app = buildApp(handlers);
const res = await app.inject({
method: "POST",
url: "/admin/api/bands/1/resolve-conflict",
headers: auth(),
payload: { field: "name = 'pwned', enriched", action: "accept_crawler" },
});
expect(res.statusCode).toBe(400);
expect(pool.find("UPDATE bands SET")).toBeUndefined();
});
it("refuse une action inconnue", async () => {
const app = buildApp(conflictPool());
const res = await app.inject({
method: "POST",
url: "/admin/api/bands/1/resolve-conflict",
headers: auth(),
payload: { field: "name", action: "drop_everything" },
});
expect(res.statusCode).toBe(400);
});
it("keep_mine ne touche que crawler_pending", async () => {
const handlers = conflictPool();
const app = buildApp(handlers);
await app.inject({
method: "POST",
url: "/admin/api/bands/1/resolve-conflict",
headers: auth(),
payload: { field: "name", action: "keep_mine" },
});
const upd = pool.find("UPDATE bands SET");
expect(upd.sql).toContain("crawler_pending = crawler_pending - $2");
expect(upd.sql).not.toContain("locked_fields");
});
it("accept_crawler applique la valeur et déverrouille", async () => {
const handlers = conflictPool();
const app = buildApp(handlers);
await app.inject({
method: "POST",
url: "/admin/api/bands/1/resolve-conflict",
headers: auth(),
payload: { field: "name", action: "accept_crawler" },
});
const upd = pool.find("UPDATE bands SET");
expect(upd.sql).toContain("name = $2::text");
expect(upd.sql).toContain("locked_fields = locked_fields - $3");
expect(upd.values[1]).toBe("MA");
});
// Régression : le cast `::text` en dur cassait les colonnes numériques.
it("utilise un cast numérique pour formed_year", async () => {
const handlers = conflictPool();
const app = buildApp(handlers);
await app.inject({
method: "POST",
url: "/admin/api/bands/1/resolve-conflict",
headers: auth(),
payload: { field: "formed_year", action: "accept_crawler" },
});
expect(pool.find("UPDATE bands SET").sql).toContain("formed_year = $2::int");
});
it("utilise un cast flottant pour lat/lon", async () => {
const handlers = conflictPool();
const app = buildApp(handlers);
await app.inject({
method: "POST",
url: "/admin/api/bands/1/resolve-conflict",
headers: auth(),
payload: { field: "lat", action: "accept_crawler" },
});
expect(pool.find("UPDATE bands SET").sql).toContain("lat = $2::double precision");
});
});
describe("POST /admin/api/job-triggers", () => {
const jobPool = () =>
([
{ match: "INSERT INTO job_triggers", result: rows({ id: 42 }) },
{ match: "INSERT INTO crawl_log", result: rows() },
]);
it.each(["enrich", "incremental", "full_crawl", "geocoder_enqueue"])(
"accepte le job_type %s",
async (job_type) => {
const app = buildApp(jobPool());
const res = await app.inject({
method: "POST",
url: "/admin/api/job-triggers",
headers: auth(),
payload: { job_type },
});
expect(res.statusCode).toBe(200);
expect(res.json().id).toBe(42);
}
);
it.each([["inconnu"], [""], ["DROP TABLE bands"]])("refuse le job_type %s", async (job_type) => {
const handlers = jobPool();
const app = buildApp(handlers);
const res = await app.inject({
method: "POST",
url: "/admin/api/job-triggers",
headers: auth(),
payload: { job_type },
});
expect(res.statusCode).toBe(400);
expect(pool.find("INSERT INTO job_triggers")).toBeUndefined();
});
it("attribue le job à l'utilisateur de la session", async () => {
const handlers = jobPool();
const app = buildApp(handlers);
await app.inject({
method: "POST",
url: "/admin/api/job-triggers",
headers: auth(),
payload: { job_type: "enrich", requested_by: "quelquun-dautre" },
});
expect(pool.find("INSERT INTO job_triggers").values).toEqual(["enrich", "nico"]);
});
});
describe("annulations", () => {
it("404 quand le run n'est plus en cours", async () => {
const handlers = ([{ match: "UPDATE crawl_run", result: rows() }]);
const app = buildApp(handlers);
const res = await app.inject({
method: "POST",
url: "/admin/api/crawl-runs/7/cancel",
headers: auth(),
payload: {},
});
expect(res.statusCode).toBe(404);
});
it("cleanup borne older_than_minutes à 1 minimum", async () => {
const handlers = ([
{ match: "UPDATE crawl_run", result: rows() },
{ match: "INSERT INTO admin_audit_log", result: rows() },
]);
const app = buildApp(handlers);
await app.inject({
method: "POST",
url: "/admin/api/crawl-runs/cleanup",
headers: auth(),
payload: { older_than_minutes: -999 },
});
expect(pool.find("UPDATE crawl_run").values).toEqual([1]);
});
});

View file

@ -0,0 +1,433 @@
import { describe, it, expect, beforeAll, afterAll } from "vitest";
process.env.ADMIN_JWT_SECRET = "secret-de-test-suffisamment-long-pour-passer-32";
const { signAdminSession, ADMIN_COOKIE_NAME } = await import("../src/adminAuth.js");
const { rows } = await import("./helpers/fakePool.js");
const { createAdminApp } = await import("./helpers/testApp.js");
let auth, app, pool;
// Une seule construction d'app pour tout le fichier ; chaque test reprogramme
// le pool via buildApp(handlers).
beforeAll(async () => {
auth = { cookie: `${ADMIN_COOKIE_NAME}=${signAdminSession("nico")}` };
({ app, pool } = await createAdminApp());
});
afterAll(async () => { await app.close(); });
/** Reprogramme le pool partagé et renvoie l'app. */
function buildApp(handlers) {
pool.reset(Array.isArray(handlers) ? handlers : handlers ?? []);
return app;
}
/** Pool permissif : n'importe quelle requête renvoie une ligne plausible. */
function anyPool(extra = []) {
return [...extra, { match: /./, result: rows({ n: 1, total: 1, id: 1, status: "done", count: 1 }) }];
}
// ------------------------------------------------------------------
// Routes de lecture : forme de la réponse + protection
// ------------------------------------------------------------------
describe("routes de lecture", () => {
const READ_ROUTES = [
"/admin/api/stats",
"/admin/api/queue",
"/admin/api/crawl-checkpoints",
"/admin/api/logs",
"/admin/api/geocoding",
"/admin/api/llm",
"/admin/api/job-triggers",
"/admin/api/live",
"/admin/api/activity",
"/admin/api/bands",
];
it.each(READ_ROUTES)("%s exige une session", async (url) => {
const handlers = ([]);
const app = buildApp(handlers);
expect((await app.inject({ method: "GET", url })).statusCode).toBe(401);
expect(pool.calls).toHaveLength(0);
});
it.each(READ_ROUTES)("%s répond 200 et ok:true avec une session", async (url) => {
const app = buildApp(anyPool());
const res = await app.inject({ method: "GET", url, headers: auth });
expect(res.statusCode).toBe(200);
expect(res.json().ok).toBe(true);
});
it.each(READ_ROUTES)("%s renvoie 500 sans divulguer l'erreur si la base tombe", async (url) => {
const handlers = ([{ match: /./, throws: new Error("FATAL: password authentication failed for user bm") }]);
const app = buildApp(handlers);
const res = await app.inject({ method: "GET", url, headers: auth });
expect(res.statusCode).toBe(500);
expect(res.body).not.toMatch(/password|user bm/);
});
});
describe("GET /admin/api/stats", () => {
it("agrège totaux, statuts, pays et genres", async () => {
const handlers = ([
{ match: "count(*)::int AS total,", result: rows({ total: 10, enriched: 4, geocoded: 6 }) },
{ match: "AS status, count", result: rows({ status: "Active", total: 7 }) },
{ match: "AS country, count", result: rows({ country: "NO", total: 3 }) },
{ match: "SELECT genre, count", result: rows({ genre: "Black Metal", total: 2 }) },
]);
const app = buildApp(handlers);
const body = (await app.inject({ method: "GET", url: "/admin/api/stats", headers: auth })).json();
expect(body.totals.total).toBe(10);
expect(body.by_status[0].status).toBe("Active");
expect(body.by_country[0].country).toBe("NO");
expect(body.by_genre[0].genre).toBe("Black Metal");
});
it("les quatre agrégats sont lancés en parallèle, pas en cascade", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/stats", headers: auth });
expect(pool.calls.length).toBe(4);
});
});
describe("GET /admin/api/bands/:ma_id", () => {
const detailPool = () =>
([
{ match: "SELECT * FROM bands WHERE ma_id", result: rows({ ma_id: 1, name: "Mayhem" }) },
{ match: "FROM band_locations", result: rows({ id: 9, step_order: 0, location_raw: "Oslo" }) },
{ match: "FROM llm_cache", result: rows({ id: 3, model: "llama-3.3-70b-versatile" }) },
]);
it("renvoie le groupe, ses localisations et ses appels LLM", async () => {
const app = buildApp(detailPool());
const body = (await app.inject({ method: "GET", url: "/admin/api/bands/1", headers: auth })).json();
expect(body.item.name).toBe("Mayhem");
expect(body.locations).toHaveLength(1);
expect(body.llm).toHaveLength(1);
});
it("répond 400 pour un ma_id invalide", async () => {
const app = buildApp(detailPool());
for (const bad of ["abc", "-1"]) {
expect((await app.inject({ method: "GET", url: `/admin/api/bands/${bad}`, headers: auth })).statusCode).toBe(400);
}
});
it("répond 404 quand le groupe n'existe pas", async () => {
const handlers = ([{ match: "SELECT * FROM bands WHERE ma_id", result: rows() }]);
const app = buildApp(handlers);
expect((await app.inject({ method: "GET", url: "/admin/api/bands/1", headers: auth })).statusCode).toBe(404);
});
// Les tables du nouveau pipeline peuvent ne pas exister sur une base ancienne :
// le détail du groupe doit rester consultable malgré tout.
it("dégrade proprement si band_locations ou llm_cache sont absentes", async () => {
const handlers = ([
{ match: "SELECT * FROM bands WHERE ma_id", result: rows({ ma_id: 1, name: "Mayhem" }) },
{ match: "FROM band_locations", throws: new Error('relation "band_locations" does not exist') },
{ match: "FROM llm_cache", throws: new Error('relation "llm_cache" does not exist') },
]);
const app = buildApp(handlers);
const res = await app.inject({ method: "GET", url: "/admin/api/bands/1", headers: auth });
expect(res.statusCode).toBe(200);
expect(res.json().locations).toEqual([]);
expect(res.json().llm).toEqual([]);
});
});
describe("GET /admin/api/logs — filtres", () => {
it("filtre par niveau, run et curseur, en paramètres liés", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({
method: "GET",
url: "/admin/api/logs?level=error&run_id=12&min_id=100&pageSize=25",
headers: auth,
});
const call = pool.find("FROM crawl_log");
expect(call.sql).toContain("level = $1");
expect(call.sql).toContain("run_id = $2");
expect(call.sql).toContain("id > $3");
expect(call.values).toEqual(["error", 12, 100, 25, 0]);
});
it("sans filtre, aucune clause WHERE", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/logs", headers: auth });
expect(pool.find("FROM crawl_log").sql).not.toContain("WHERE");
});
// Régression : page/pageSize étaient calculés mais jamais renvoyés, le client
// ne pouvait donc pas savoir s'il restait des logs à charger.
it("renvoie les informations de pagination", async () => {
const app = buildApp(anyPool());
const body = (await app.inject({ method: "GET", url: "/admin/api/logs?page=3", headers: auth })).json();
expect(body.page).toBe(3);
expect(body.pageSize).toBe(50);
});
});
describe("GET /admin/api/llm — filtres", () => {
it("filtre par modèle, nullité et texte", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({
method: "GET",
url: "/admin/api/llm?model=llama-3.1-8b-instant&only_null=1&q=oslo",
headers: auth,
});
const call = pool.find("FROM llm_cache lc");
expect(call.sql).toContain("model = $1");
expect(call.sql).toContain("is_null = true");
expect(call.values).toContain("llama-3.1-8b-instant");
expect(call.values).toContain("%oslo%");
});
it("le count applique le même filtre que la page", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/llm?model=x", headers: auth });
const count = pool.find("count(*)::int AS total FROM llm_cache");
expect(count.sql).toContain("model = $1");
expect(count.values).toEqual(["x"]);
});
it("tronque une recherche exagérément longue", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: `/admin/api/llm?q=${"a".repeat(500)}`, headers: auth });
const bound = pool.find("FROM llm_cache lc").values.find((v) => String(v).startsWith("%aaa"));
expect(bound).toHaveLength(102); // 100 caractères + les deux %
});
});
describe("GET /admin/api/activity — liste unifiée", () => {
it("réunit crawl_run et admin_audit_log", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/activity", headers: auth });
const call = pool.find("UNION ALL");
expect(call.sql).toContain("FROM crawl_run");
expect(call.sql).toContain("FROM admin_audit_log");
});
it("filtre par type et statut en paramètres liés", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/activity?type=run&status=running", headers: auth });
const data = pool.find(/SELECT \* FROM \([\s\S]*UNION ALL/);
expect(data.values).toEqual(["run", "running", 50, 0]);
});
it("trie par date décroissante", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/activity", headers: auth });
expect(pool.find(/SELECT \* FROM \(/).sql).toContain("ORDER BY ts DESC");
});
});
// ------------------------------------------------------------------
// Actions de maintenance du géocodage
// ------------------------------------------------------------------
describe("actions sur band_locations", () => {
const ACTIONS = [
["/admin/api/locations/reset-errors", "geocode_status = 'error'"],
["/admin/api/locations/reset-llm", "geocode_status IN ('llm_needed', 'manual')"],
["/admin/api/locations/reset-all", "is_country_only = FALSE"],
["/admin/api/locations/requeue-all", "geocode_status = ANY($1::text[])"],
];
it.each(ACTIONS.map(([url]) => url))("%s exige une session", async (url) => {
const handlers = ([]);
const app = buildApp(handlers);
expect((await app.inject({ method: "POST", url, payload: {} })).statusCode).toBe(401);
expect(pool.calls).toHaveLength(0);
});
it.each(ACTIONS)("%s cible les bonnes lignes", async (url, expectedClause) => {
const handlers = ([
{ match: "UPDATE band_locations", result: { rows: [], rowCount: 12 } },
{ match: "INSERT INTO crawl_log", result: rows() },
]);
const app = buildApp(handlers);
const res = await app.inject({ method: "POST", url, headers: auth, payload: {} });
expect(res.statusCode).toBe(200);
expect(res.json().count).toBe(12);
expect(pool.find("UPDATE band_locations").sql).toContain(expectedClause);
});
it.each(ACTIONS.map(([url]) => url))("%s remet les lignes en queue", async (url) => {
const handlers = ([
{ match: "UPDATE band_locations", result: { rows: [], rowCount: 1 } },
{ match: "INSERT INTO crawl_log", result: rows() },
]);
const app = buildApp(handlers);
await app.inject({ method: "POST", url, headers: auth, payload: {} });
expect(pool.find("UPDATE band_locations").sql).toContain("geocode_status='queued'");
});
it.each(ACTIONS.map(([url]) => url))("%s trace l'action dans les logs avec l'auteur", async (url) => {
const handlers = ([
{ match: "UPDATE band_locations", result: { rows: [], rowCount: 1 } },
{ match: "INSERT INTO crawl_log", result: rows() },
]);
const app = buildApp(handlers);
await app.inject({ method: "POST", url, headers: auth, payload: {} });
expect(pool.find("INSERT INTO crawl_log").values[1]).toContain("admin:nico");
});
// reset-all efface les coordonnées (le worker va tout re-géocoder) alors que
// reset-errors ne remet en file que ce qui avait échoué.
it("reset-all efface les coordonnées, reset-errors non", async () => {
const mk = async (url) => {
const handlers = ([
{ match: "UPDATE band_locations", result: { rows: [], rowCount: 1 } },
{ match: "INSERT INTO crawl_log", result: rows() },
]);
const app = buildApp(handlers);
await app.inject({ method: "POST", url, headers: auth, payload: {} });
return pool.find("UPDATE band_locations").sql;
};
expect(await mk("/admin/api/locations/reset-all")).toContain("lat=NULL");
expect(await mk("/admin/api/locations/reset-errors")).not.toContain("lat=NULL");
});
it("requeue-all inclut 'done' seulement sur demande explicite", async () => {
const mk = async (payload) => {
const handlers = ([
{ match: "UPDATE band_locations", result: { rows: [], rowCount: 1 } },
{ match: "INSERT INTO crawl_log", result: rows() },
]);
const app = buildApp(handlers);
await app.inject({ method: "POST", url: "/admin/api/locations/requeue-all", headers: auth, payload });
return pool.find("UPDATE band_locations").values[0];
};
expect(await mk({})).toEqual(["error", "llm_needed"]);
expect(await mk({ include_done: true })).toEqual(["error", "llm_needed", "manual", "done"]);
});
it("requeue-all épargne toujours les localisations pays-seul", async () => {
const handlers = ([
{ match: "UPDATE band_locations", result: { rows: [], rowCount: 1 } },
{ match: "INSERT INTO crawl_log", result: rows() },
]);
const app = buildApp(handlers);
await app.inject({
method: "POST", url: "/admin/api/locations/requeue-all",
headers: auth, payload: { include_done: true },
});
expect(pool.find("UPDATE band_locations").sql).toContain("is_country_only = FALSE");
});
it("une action qui échoue renvoie 500 sans écrire de log de succès", async () => {
const handlers = ([{ match: "UPDATE band_locations", throws: new Error("deadlock") }]);
const app = buildApp(handlers);
const res = await app.inject({
method: "POST", url: "/admin/api/locations/reset-errors", headers: auth, payload: {},
});
expect(res.statusCode).toBe(500);
expect(pool.find("INSERT INTO crawl_log")).toBeUndefined();
});
});
describe("POST /admin/api/geocode-cache/purge-nominatim", () => {
it("ne supprime que les entrées de l'ancien pipeline", async () => {
const handlers = ([
{ match: "DELETE FROM geocode_cache", result: { rows: [], rowCount: 4 } },
{ match: "INSERT INTO crawl_log", result: rows() },
]);
const app = buildApp(handlers);
const res = await app.inject({
method: "POST", url: "/admin/api/geocode-cache/purge-nominatim", headers: auth, payload: {},
});
expect(res.json().count).toBe(4);
// Le marqueur place_rank distingue Nominatim de Geoapify : sans ce WHERE,
// la purge viderait aussi le cache Geoapify (appels payants).
expect(pool.find("DELETE FROM geocode_cache").sql).toContain("place_rank");
});
it("exige une session", async () => {
const handlers = ([]);
const app = buildApp(handlers);
expect((await app.inject({
method: "POST", url: "/admin/api/geocode-cache/purge-nominatim", payload: {},
})).statusCode).toBe(401);
expect(pool.calls).toHaveLength(0);
});
});
describe("POST /admin/api/crawl-runs/cleanup", () => {
const cleanupPool = () =>
([
{ match: "UPDATE crawl_run", result: rows({ id: 1, run_type: "enrich", started_at: "2026-01-01" }) },
{ match: "INSERT INTO admin_audit_log", result: rows() },
]);
it("utilise 30 minutes par défaut", async () => {
const handlers = cleanupPool();
const app = buildApp(handlers);
await app.inject({ method: "POST", url: "/admin/api/crawl-runs/cleanup", headers: auth, payload: {} });
expect(pool.find("UPDATE crawl_run").values).toEqual([30]);
});
it("ne touche que les runs marqués running", async () => {
const handlers = cleanupPool();
const app = buildApp(handlers);
await app.inject({ method: "POST", url: "/admin/api/crawl-runs/cleanup", headers: auth, payload: {} });
expect(pool.find("UPDATE crawl_run").sql).toContain("status = 'running'");
});
it("renvoie le nombre de runs nettoyés", async () => {
const app = buildApp(cleanupPool());
const res = await app.inject({
method: "POST", url: "/admin/api/crawl-runs/cleanup", headers: auth, payload: { older_than_minutes: 60 },
});
expect(res.json().cleaned).toBe(1);
});
});
describe("GET /admin/api/geocoding", () => {
it("dégrade proprement quand les tables du nouveau pipeline manquent", async () => {
const handlers = ([
{ match: "FROM geocode_cache", result: rows({ n: 5 }) },
{ match: "FROM bands b", result: rows() },
{ match: "FROM band_locations", throws: new Error("relation does not exist") },
{ match: "FROM llm_cache", throws: new Error("relation does not exist") },
]);
const app = buildApp(handlers);
const res = await app.inject({ method: "GET", url: "/admin/api/geocoding", headers: auth });
expect(res.statusCode).toBe(200);
expect(res.json().cache_size).toBe(5);
expect(res.json().locations).toEqual([]);
});
});
describe("GET /admin/api/live", () => {
it("remonte runs actifs, jobs en attente et file de géocodage", async () => {
const handlers = ([
{ match: "WHERE status = 'running'", result: rows({ id: 1, run_type: "enrich" }) },
{ match: "WHERE status = 'pending'", result: rows({ id: 2, job_type: "enrich" }) },
{ match: "GROUP BY geocode_status", result: rows({ status: "queued", n: 42 }) },
{ match: "geocode_status = 'processing'", result: rows() },
{ match: "FROM crawl_checkpoint", result: rows({ key: "last_full_crawl_at" }) },
]);
const app = buildApp(handlers);
const body = (await app.inject({ method: "GET", url: "/admin/api/live", headers: auth })).json();
expect(body.active_runs).toHaveLength(1);
expect(body.pending_jobs).toHaveLength(1);
expect(body.geo_queue[0].n).toBe(42);
expect(body.checkpoints).toHaveLength(1);
});
it("limite l'aperçu des localisations en cours de traitement", async () => {
const handlers = anyPool();
const app = buildApp(handlers);
await app.inject({ method: "GET", url: "/admin/api/live", headers: auth });
expect(pool.find("geocode_status = 'processing'").sql).toContain("LIMIT 3");
});
});

View file

@ -0,0 +1,252 @@
import { describe, it, expect, beforeAll, afterAll } from "vitest";
process.env.ADMIN_JWT_SECRET = "secret-de-test-suffisamment-long-pour-passer-32";
const { signAdminSession, ADMIN_COOKIE_NAME } = await import("../src/adminAuth.js");
const { rows } = await import("./helpers/fakePool.js");
const { createAdminApp } = await import("./helpers/testApp.js");
let auth, app, pool;
beforeAll(async () => {
auth = { cookie: `${ADMIN_COOKIE_NAME}=${signAdminSession("nico")}` };
({ app, pool } = await createAdminApp());
});
afterAll(async () => { await app.close(); });
/** Reprogramme le pool partagé et renvoie l'app (construite une seule fois). */
function buildApp(handlers) {
pool.reset(handlers ?? []);
return app;
}
/**
* L'annulation d'un run est COOPÉRATIVE : l'API pose un drapeau, le crawler
* l'observe et s'arrête. Ces tests verrouillent le contrat côté API le
* pendant côté crawler est dans apps/crawler/tests/test_cancellation.py.
*/
describe("POST /admin/api/crawl-runs/:id/cancel", () => {
const runningPool = () =>
([
{ match: "SET cancel_requested = TRUE", result: rows({ id: 7, run_type: "full_europe" }) },
{ match: "INSERT INTO crawl_log", result: rows() },
{ match: "INSERT INTO admin_audit_log", result: rows() },
]);
it("pose le drapeau sans toucher au statut", async () => {
const handlers = runningPool();
const app = buildApp(handlers);
const res = await app.inject({
method: "POST", url: "/admin/api/crawl-runs/7/cancel", headers: auth, payload: {},
});
expect(res.statusCode).toBe(200);
expect(res.json().cancel_requested).toBe(true);
const upd = pool.find("UPDATE crawl_run");
expect(upd.sql).toContain("cancel_requested = TRUE");
// Le point central de la correction : l'API ne décide plus du statut.
// C'est le crawler qui écrira 'cancelled' quand il se sera vraiment arrêté.
// On n'inspecte que la clause SET — `status` a le droit d'apparaître dans
// le WHERE (on ne cible que les runs en cours).
const setClause = upd.sql.split(/\bWHERE\b/i)[0];
expect(setClause).not.toMatch(/status\s*=/);
expect(setClause).not.toContain("finished_at");
});
it("ne cible que les runs réellement en cours", async () => {
const handlers = runningPool();
const app = buildApp(handlers);
await app.inject({ method: "POST", url: "/admin/api/crawl-runs/7/cancel", headers: auth, payload: {} });
expect(pool.find("UPDATE crawl_run").sql).toContain("status = 'running'");
});
it("enregistre qui a demandé l'annulation", async () => {
const handlers = runningPool();
const app = buildApp(handlers);
await app.inject({ method: "POST", url: "/admin/api/crawl-runs/7/cancel", headers: auth, payload: {} });
expect(pool.find("UPDATE crawl_run").values).toEqual([7, "nico"]);
expect(pool.find("INSERT INTO admin_audit_log")).toBeDefined();
});
it("est idempotent : une seconde demande répond 200, pas une erreur", async () => {
const handlers = ([
{ match: "SET cancel_requested = TRUE", result: rows() }, // 0 ligne : déjà demandé
{ match: "SELECT status, cancel_requested", result: rows({ status: "running", cancel_requested: true }) },
]);
const app = buildApp(handlers);
const res = await app.inject({
method: "POST", url: "/admin/api/crawl-runs/7/cancel", headers: auth, payload: {},
});
expect(res.statusCode).toBe(200);
expect(res.json().already_requested).toBe(true);
});
it("répond 404 pour un run terminé", async () => {
const handlers = ([
{ match: "SET cancel_requested = TRUE", result: rows() },
{ match: "SELECT status, cancel_requested", result: rows({ status: "done", cancel_requested: false }) },
]);
const app = buildApp(handlers);
const res = await app.inject({
method: "POST", url: "/admin/api/crawl-runs/7/cancel", headers: auth, payload: {},
});
expect(res.statusCode).toBe(404);
});
it("exige une session admin", async () => {
const handlers = ([]);
const app = buildApp(handlers);
const res = await app.inject({ method: "POST", url: "/admin/api/crawl-runs/7/cancel", payload: {} });
expect(res.statusCode).toBe(401);
expect(pool.calls).toHaveLength(0);
});
});
describe("POST /admin/api/job-triggers/:id/cancel", () => {
it("retire un job encore en file", async () => {
const handlers = ([
{ match: "UPDATE job_triggers", result: rows({ id: 3, job_type: "enrich" }) },
{ match: "INSERT INTO crawl_log", result: rows() },
]);
const app = buildApp(handlers);
const res = await app.inject({ method: "POST", url: "/admin/api/job-triggers/3/cancel", headers: auth, payload: {} });
expect(res.statusCode).toBe(200);
expect(pool.find("UPDATE job_triggers").sql).toContain("status='pending'");
});
// Un job déjà réclamé n'existe plus qu'à travers son crawl_run : le message
// doit y renvoyer explicitement au lieu d'un « non trouvé » trompeur.
it("répond 409 avec un message actionnable si le job a déjà démarré", async () => {
const handlers = ([
{ match: "UPDATE job_triggers", result: rows() },
{ match: "SELECT status FROM job_triggers", result: rows({ status: "running" }) },
]);
const app = buildApp(handlers);
const res = await app.inject({ method: "POST", url: "/admin/api/job-triggers/3/cancel", headers: auth, payload: {} });
expect(res.statusCode).toBe(409);
expect(res.json().error).toMatch(/déjà démarré/);
expect(res.json().error).toMatch(/run/i);
});
it("répond 409 « non trouvé » pour un job inexistant", async () => {
const handlers = ([
{ match: "UPDATE job_triggers", result: rows() },
{ match: "SELECT status FROM job_triggers", result: rows() },
]);
const app = buildApp(handlers);
const res = await app.inject({ method: "POST", url: "/admin/api/job-triggers/99/cancel", headers: auth, payload: {} });
expect(res.json().error).toMatch(/non trouvé/);
});
});
/**
* L'édition admin de lat/lon doit atteindre band_locations : c'est cette table
* que la carte clusterise. Sans ça, corriger un point n'avait aucun effet visible.
*/
describe("PATCH /admin/api/bands/:ma_id — cohérence géographique", () => {
const patchPool = (bandAfter) =>
([
{ match: "SELECT * FROM bands WHERE ma_id", result: rows({ ma_id: 1, name: "Avant" }) },
{ match: "UPDATE bands SET", result: rows(bandAfter) },
{ match: "INSERT INTO band_locations", result: rows() },
{ match: "DELETE FROM band_locations", result: rows() },
{ match: "INSERT INTO admin_audit_log", result: rows() },
]);
it("pose un override dans band_locations quand lat/lon sont fournis", async () => {
const handlers = patchPool({ ma_id: 1, lat: 59.9, lon: 10.7, location_text: "Oslo" });
const app = buildApp(handlers);
await app.inject({
method: "PATCH", url: "/admin/api/bands/1", headers: auth,
payload: { lat: 59.9, lon: 10.7 },
});
const ins = pool.find("INSERT INTO band_locations");
expect(ins).toBeDefined();
expect(ins.values).toEqual([1, "Oslo", 59.9, 10.7]);
// step_order = -1 → l'override passe avant tous les steps parsés et devient
// le point principal repris par _sync_band_point côté geocoder.
expect(ins.sql).toContain("-1");
expect(ins.sql).toContain("'done'");
expect(ins.sql).toContain("'admin'");
});
it("l'override est un upsert, pas un doublon", async () => {
const handlers = patchPool({ ma_id: 1, lat: 1, lon: 2, location_text: "X" });
const app = buildApp(handlers);
await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth, payload: { lat: 1, lon: 2 } });
expect(pool.find("INSERT INTO band_locations").sql).toContain("ON CONFLICT (ma_id, step_order, location_raw) DO UPDATE");
});
it("efface l'override quand lat/lon sont vidés", async () => {
const handlers = patchPool({ ma_id: 1, lat: null, lon: null, location_text: "Oslo" });
const app = buildApp(handlers);
await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth, payload: { lat: null, lon: null } });
const del = pool.find("DELETE FROM band_locations");
expect(del).toBeDefined();
expect(del.sql).toContain("step_order = -1");
expect(pool.find("INSERT INTO band_locations")).toBeUndefined();
});
it("ne touche pas band_locations quand l'édition ne concerne pas les coordonnées", async () => {
const handlers = patchPool({ ma_id: 1, name: "Après" });
const app = buildApp(handlers);
await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth, payload: { name: "Après" } });
expect(pool.find("INSERT INTO band_locations")).toBeUndefined();
expect(pool.find("DELETE FROM band_locations")).toBeUndefined();
});
});
describe("PATCH /admin/api/bands/:ma_id — transaction", () => {
const okPool = () =>
([
{ match: "SELECT * FROM bands WHERE ma_id", result: rows({ ma_id: 1, name: "Avant" }) },
{ match: "UPDATE bands SET", result: rows({ ma_id: 1, name: "Après" }) },
{ match: "INSERT INTO admin_audit_log", result: rows() },
]);
it("encadre l'écriture et l'audit dans une même transaction", async () => {
const handlers = okPool();
const app = buildApp(handlers);
await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth, payload: { name: "Après" } });
expect(pool.sequence(["BEGIN", "UPDATE bands SET", "INSERT INTO admin_audit_log", "COMMIT"]))
.toEqual(["BEGIN", "UPDATE bands SET", "INSERT INTO admin_audit_log", "COMMIT"]);
expect(pool.rolledBack).toBe(false);
});
it("annule tout si l'écriture d'audit échoue", async () => {
const handlers = ([
{ match: "SELECT * FROM bands WHERE ma_id", result: rows({ ma_id: 1 }) },
{ match: "UPDATE bands SET", result: rows({ ma_id: 1 }) },
{ match: "INSERT INTO admin_audit_log", throws: new Error("disque plein") },
]);
const app = buildApp(handlers);
const res = await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth, payload: { name: "X" } });
expect(res.statusCode).toBe(500);
expect(pool.rolledBack).toBe(true);
expect(pool.committed).toBe(false);
});
it("libère la connexion même en cas d'erreur", async () => {
const handlers = ([
{ match: "SELECT * FROM bands WHERE ma_id", result: rows({ ma_id: 1 }) },
{ match: "UPDATE bands SET", throws: new Error("boom") },
]);
const app = buildApp(handlers);
await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth, payload: { name: "X" } });
expect(pool.released).toBe(pool.connections);
expect(pool.released).toBeGreaterThan(0);
});
it("annule la transaction quand le groupe est introuvable", async () => {
const handlers = ([{ match: "SELECT * FROM bands WHERE ma_id", result: rows() }]);
const app = buildApp(handlers);
const res = await app.inject({ method: "PATCH", url: "/admin/api/bands/1", headers: auth, payload: { name: "X" } });
expect(res.statusCode).toBe(404);
expect(pool.rolledBack).toBe(true);
expect(pool.released).toBe(1);
});
});

View file

@ -0,0 +1,145 @@
import { describe, it, expect } from "vitest";
import { makeFakePool, rows } from "./helpers/fakePool.js";
/**
* Méta-tests du harnais.
*
* Presque toutes les assertions du projet s'appuient sur `makeFakePool` :
* si ce double se met à tout avaler en silence (un `find` qui renvoie toujours
* undefined, un `throws` ignoré, un compteur qui ne s'incrémente pas), des
* dizaines de tests passeraient au vert sans plus rien vérifier.
*
* Ces tests vérifient donc l'outil de mesure lui-même.
*/
describe("makeFakePool — enregistrement", () => {
it("enregistre chaque requête avec son SQL et ses paramètres", async () => {
const pool = makeFakePool();
await pool.query("SELECT 1 FROM bands WHERE ma_id = $1", [42]);
expect(pool.calls).toHaveLength(1);
expect(pool.calls[0].sql).toContain("FROM bands");
expect(pool.calls[0].values).toEqual([42]);
});
it("normalise l'absence de paramètres en tableau vide", async () => {
const pool = makeFakePool();
await pool.query("SELECT 1");
expect(pool.calls[0].values).toEqual([]);
});
it("accepte la forme objet { text }", async () => {
const pool = makeFakePool();
await pool.query({ text: "SELECT 1 FROM bands" }, []);
expect(pool.calls[0].sql).toBe("SELECT 1 FROM bands");
});
});
describe("makeFakePool — recherche", () => {
it("find() trouve par sous-chaîne ET renvoie undefined quand rien ne matche", async () => {
const pool = makeFakePool();
await pool.query("UPDATE bands SET name = $1", ["x"]);
expect(pool.find("UPDATE bands")).toBeDefined();
// Le cas qui compte : un find() qui renverrait toujours undefined ferait
// passer au vert tous les `expect(pool.find(...)).toBeUndefined()`.
expect(pool.find("DELETE FROM bands")).toBeUndefined();
});
it("find() accepte une RegExp", async () => {
const pool = makeFakePool();
await pool.query("SELECT a, b FROM bands");
expect(pool.find(/SELECT\s+a,\s*b/)).toBeDefined();
expect(pool.find(/SELECT\s+z/)).toBeUndefined();
});
it("findAll() renvoie toutes les occurrences, pas seulement la première", async () => {
const pool = makeFakePool();
await pool.query("INSERT INTO bands VALUES (1)");
await pool.query("INSERT INTO bands VALUES (2)");
await pool.query("SELECT 1");
expect(pool.findAll("INSERT INTO bands")).toHaveLength(2);
expect(pool.findAll("DELETE")).toHaveLength(0);
});
it("sequence() reflète l'ordre réel d'exécution", async () => {
const pool = makeFakePool();
await pool.query("BEGIN");
await pool.query("UPDATE bands SET x = 1");
await pool.query("COMMIT");
expect(pool.sequence(["COMMIT", "BEGIN", "UPDATE bands"]))
.toEqual(["BEGIN", "UPDATE bands", "COMMIT"]);
});
});
describe("makeFakePool — handlers", () => {
it("renvoie le résultat du premier handler qui matche", async () => {
const pool = makeFakePool([
{ match: "FROM bands", result: rows({ ma_id: 1 }) },
{ match: "FROM bands", result: rows({ ma_id: 2 }) },
]);
expect((await pool.query("SELECT * FROM bands")).rows[0].ma_id).toBe(1);
});
it("renvoie un résultat vide quand aucun handler ne matche", async () => {
const pool = makeFakePool([{ match: "FROM autre", result: rows({ x: 1 }) }]);
expect(await pool.query("SELECT * FROM bands")).toEqual({ rows: [], rowCount: 0 });
});
// Si `throws` était ignoré, tous les tests de chemin d'erreur (rollback,
// 500 sans fuite d'information) passeraient sans rien exercer.
it("throws fait bien lever la requête", async () => {
const pool = makeFakePool([{ match: "FROM bands", throws: new Error("boum") }]);
await expect(pool.query("SELECT * FROM bands")).rejects.toThrow("boum");
});
it("result peut être une fonction du SQL et des paramètres", async () => {
const pool = makeFakePool([
{ match: "FROM bands", result: (_sql, values) => rows({ echo: values[0] }) },
]);
expect((await pool.query("SELECT * FROM bands WHERE id=$1", [7])).rows[0].echo).toBe(7);
});
it("rows() produit un rowCount cohérent", () => {
expect(rows()).toEqual({ rows: [], rowCount: 0 });
expect(rows({ a: 1 }, { a: 2 }).rowCount).toBe(2);
});
});
describe("makeFakePool — transactions", () => {
it("connect() renvoie un client qui partage le journal des requêtes", async () => {
const pool = makeFakePool();
const client = await pool.connect();
await client.query("SELECT 1 FROM bands");
expect(pool.find("FROM bands")).toBeDefined();
});
it("détecte COMMIT et ROLLBACK séparément", async () => {
const committed = makeFakePool();
const c1 = await committed.connect();
await c1.query("COMMIT");
expect(committed.committed).toBe(true);
expect(committed.rolledBack).toBe(false);
const rolled = makeFakePool();
const c2 = await rolled.connect();
await c2.query("ROLLBACK");
expect(rolled.rolledBack).toBe(true);
expect(rolled.committed).toBe(false);
});
it("compte les connexions ouvertes et libérées", async () => {
const pool = makeFakePool();
const client = await pool.connect();
expect(pool.connections).toBe(1);
expect(pool.released).toBe(0);
client.release();
expect(pool.released).toBe(1);
});
it("un pool neuf ne rapporte ni commit ni rollback", () => {
const pool = makeFakePool();
expect(pool.committed).toBe(false);
expect(pool.rolledBack).toBe(false);
expect(pool.connections).toBe(0);
expect(pool.released).toBe(0);
});
});

View file

@ -0,0 +1,110 @@
/**
* Faux `pg.Pool` pour les tests de routes.
*
* Objectif : tester le routage, l'authentification, la validation d'entrée et
* la *forme* du SQL généré (allowlists de colonnes, casts, transactions) sans
* démarrer de conteneur Postgres. Un test qui monte un conteneur met des
* dizaines de secondes ; celui-ci tourne en millisecondes.
*
* `handlers` associe un fragment de SQL (sous-chaîne ou RegExp) à un résultat.
* Toutes les requêtes exécutées sont enregistrées dans `pool.calls`.
*
* Le faux client rend les transactions observables : `pool.committed` et
* `pool.rolledBack` permettent d'affirmer qu'une route a bien annulé son travail
* sur erreur, et `pool.released` qu'elle n'a pas fui de connexion.
*/
/**
* @typedef {object} Handler
* @property {string|RegExp} match fragment de SQL à reconnaître
* @property {any} [result] résultat pg, ou fonction (sql, values) => résultat
* @property {Error|string} [throws] fait échouer la requête (chemins d'erreur)
*
* @param {Handler[]} [handlers]
*/
export function makeFakePool(handlers = []) {
const calls = [];
const state = { committed: false, rolledBack: false, released: 0, connections: 0 };
let active = handlers;
async function query(text, values) {
const sql = typeof text === "string" ? text : text.text;
calls.push({ sql, values: values ?? [] });
if (sql === "COMMIT") state.committed = true;
if (sql === "ROLLBACK") state.rolledBack = true;
for (const { match, result, throws } of active) {
const hit = match instanceof RegExp ? match.test(sql) : sql.includes(match);
if (hit) {
if (throws) throw throws instanceof Error ? throws : new Error(String(throws));
return typeof result === "function" ? result(sql, values) : result;
}
}
return { rows: [], rowCount: 0 };
}
const pool = {
calls,
query,
/**
* Reconfigure le pool sans en recréer un.
*
* Permet de construire l'app Fastify UNE fois par fichier de test (14 ms
* par construction, multipliées par ~12 000 exécutions pendant un run de
* mutation) au lieu d'une fois par cas de test.
*/
reset(newHandlers = []) {
active = newHandlers;
calls.length = 0;
state.committed = false;
state.rolledBack = false;
state.released = 0;
state.connections = 0;
return pool;
},
async connect() {
state.connections++;
return {
query,
release() {
state.released++;
},
};
},
get committed() { return state.committed; },
get rolledBack() { return state.rolledBack; },
get released() { return state.released; },
get connections() { return state.connections; },
/** Retrouve la première requête exécutée contenant `fragment`. */
find(fragment) {
return calls.find((c) =>
fragment instanceof RegExp ? fragment.test(c.sql) : c.sql.includes(fragment)
);
},
/** Toutes les requêtes contenant `fragment`. */
findAll(fragment) {
return calls.filter((c) =>
fragment instanceof RegExp ? fragment.test(c.sql) : c.sql.includes(fragment)
);
},
/** Ordre d'exécution des requêtes correspondant à l'un des fragments. */
sequence(fragments) {
return calls
.map((c) => fragments.find((f) => (f instanceof RegExp ? f.test(c.sql) : c.sql.includes(f))))
.filter(Boolean);
},
};
return pool;
}
/** Raccourci : un résultat pg minimal. */
export function rows(...items) {
return { rows: items, rowCount: items.length };
}

View file

@ -0,0 +1,65 @@
import Fastify from "fastify";
import cookie from "@fastify/cookie";
import { makeFakePool } from "./fakePool.js";
/**
* Applications de test partagées.
*
* Construire une app Fastify coûte ~14 ms (helmet + cookie + rate-limit +
* ready()). C'est négligeable une fois, mais les tests de mutation rejouent la
* suite des milliers de fois : à raison d'une construction par cas de test, ces
* 14 ms devenaient l'essentiel des 5 minutes du run.
*
* On construit donc l'app UNE fois par fichier, autour d'un pool reconfigurable
* (`pool.reset(handlers)`), et chaque test ne fait que reprogrammer le pool.
*
* Une app par fichier de test (et non une globale partagée) : les fichiers
* s'exécutent dans des workers séparés, et l'isolation reste totale puisque le
* seul état mutable est le pool, remis à zéro avant chaque cas.
*/
/** App n'exposant que les routes admin (plugin adminRoutes seul). */
export async function createAdminApp() {
const { default: adminRoutes } = await import("../../src/adminRoutes.js");
const pool = makeFakePool();
const app = Fastify({ logger: false });
await app.register(cookie);
await app.register(adminRoutes, { pool });
await app.ready();
return { app, pool };
}
/** App complète (routes publiques + admin), via la fabrique de production. */
export async function createFullApp(opts = {}) {
const { buildServer } = await import("../../src/app.js");
const pool = makeFakePool();
const app = await buildServer({
pool,
logger: false,
corsOrigins: ["https://metalfrom.eu"],
seedAdmin: false,
// L'app est partagée entre les cas du fichier : les plafonds de débit
// réels rendraient le 11e test rouge sans rapport avec ce qu'il vérifie.
// Les vraies valeurs sont testées dans rateLimit.test.js.
globalRateLimitMax: 1e9,
adminRateLimitMax: 1e9,
authRateLimitMax: 1e9,
...opts,
});
return { app, pool };
}
/**
* Quelques cas doivent construire leur propre app (options de fabrique
* différentes : jeton d'import absent, pool null). Ils restent rares et
* assument le coût de construction.
*/
export async function createFullAppWith(opts) {
const { buildServer } = await import("../../src/app.js");
return buildServer({
logger: false,
corsOrigins: ["https://metalfrom.eu"],
seedAdmin: false,
...opts,
});
}

View file

@ -0,0 +1,430 @@
import { describe, it, expect, beforeAll, afterAll } from "vitest";
process.env.ADMIN_JWT_SECRET = "secret-de-test-suffisamment-long-pour-passer-32";
const { rows } = await import("./helpers/fakePool.js");
const { createFullApp, createFullAppWith } = await import("./helpers/testApp.js");
const TOKEN = "c4b99106fcd09badce25810e974ec4d6";
// Deux apps pour tout le fichier (cf. helpers/testApp.js : construire une app
// coûte ~14 ms, multipliées par des milliers d'exécutions en tests de mutation).
// Chaque cas se contente de reprogrammer le pool associé.
let app, pool, tokenApp, tokenPool;
beforeAll(async () => {
({ app, pool } = await createFullApp());
({ app: tokenApp, pool: tokenPool } = await createFullApp({ importToken: TOKEN }));
});
afterAll(async () => {
await app.close();
await tokenApp.close();
});
/** Handlers par défaut : chaque requête renvoie quelque chose de plausible. */
function defaultHandlers() {
return [
{ match: "FROM band_locations bl", result: rows({ ma_id: 1, name: "Mayhem", lat: 59.9, lon: 10.7 }) },
{ match: "grid_cells", result: rows({ lat: 59.9, lon: 10.7, count: 3, sample_bands: [] }) },
{ match: "FROM bands", result: rows({ ma_id: 1, name: "Mayhem", total: 1 }) },
];
}
/**
* Reprogramme le pool partagé et renvoie l'app publique.
* @param {import("./helpers/fakePool.js").Handler[]} [handlers]
*/
function build(handlers = defaultHandlers()) {
pool.reset(handlers);
return app;
}
/** Idem pour l'app configurée avec un jeton d'import. */
function buildToken(handlers = []) {
tokenPool.reset(handlers);
return tokenApp;
}
describe("/api/health et /", () => {
it("health répond 200 sans toucher la base", async () => {
const res = await build([]).inject({ method: "GET", url: "/api/health" });
expect(res.statusCode).toBe(200);
expect(res.json()).toEqual({ ok: true });
expect(pool.calls).toHaveLength(0);
});
it("la racine identifie le service", async () => {
expect((await build([]).inject({ method: "GET", url: "/" })).json().ok).toBe(true);
});
});
describe("CORS", () => {
it("renvoie l'en-tête pour une origine autorisée", async () => {
const res = await build().inject({
method: "GET", url: "/api/health", headers: { origin: "https://metalfrom.eu" },
});
expect(res.headers["access-control-allow-origin"]).toBe("https://metalfrom.eu");
});
it("n'écho PAS une origine non autorisée", async () => {
const res = await build().inject({
method: "GET", url: "/api/health", headers: { origin: "https://evil.example" },
});
expect(res.headers["access-control-allow-origin"]).toBeUndefined();
});
it("ne fait pas de match par préfixe sur l'origine", async () => {
const res = await build().inject({
method: "GET", url: "/api/health", headers: { origin: "https://metalfrom.eu.evil.example" },
});
expect(res.headers["access-control-allow-origin"]).toBeUndefined();
});
// Régression : sans `return reply`, Fastify poursuivait le cycle de vie et le
// préflight repartait dans le handler de route (ou un 404).
it("le préflight OPTIONS répond 204 et court-circuite le routage", async () => {
const res = await build().inject({ method: "OPTIONS", url: "/api/bands" });
expect(res.statusCode).toBe(204);
expect(res.body).toBe("");
});
it("OPTIONS répond 204 même sur une route inexistante", async () => {
expect((await build().inject({ method: "OPTIONS", url: "/nimporte/quoi" })).statusCode).toBe(204);
});
});
describe("en-têtes de sécurité (helmet)", () => {
it.each([
"x-content-type-options",
"x-frame-options",
"strict-transport-security",
])("pose %s", async (header) => {
const res = await build().inject({ method: "GET", url: "/api/health" });
expect(res.headers[header]).toBeDefined();
});
it("ne divulgue pas la stack ni les identifiants en cas d'erreur serveur", async () => {
const res = await build([
{ match: "FROM bands", throws: new Error("connexion perdue: host=10.0.1.7 user=bm") },
]).inject({ method: "GET", url: "/api/stats" });
expect(res.statusCode).toBe(500);
expect(res.body).not.toMatch(/10\.0\.1\.7|user=bm|at Object/);
});
});
describe("/api/bands — validation", () => {
it("répond 200 sans paramètre", async () => {
const res = await build().inject({ method: "GET", url: "/api/bands" });
expect(res.statusCode).toBe(200);
expect(res.json().limit).toBe(1000);
});
// Régression : Number("abc") = NaN partait dans LIMIT $n → erreur pg → 500.
it.each([
["limit=abc", "/api/bands?limit=abc"],
["limit=0", "/api/bands?limit=0"],
["limit=-5", "/api/bands?limit=-5"],
["offset=abc", "/api/bands?offset=abc"],
["offset=-1", "/api/bands?offset=-1"],
["offset>1M", "/api/bands?offset=1000001"],
])("répond 400 (pas 500) pour %s", async (_label, url) => {
const res = await build().inject({ method: "GET", url });
expect(res.statusCode).toBe(400);
expect(res.json().ok).toBe(false);
});
it("plafonne le limit sans erreur", async () => {
expect((await build().inject({ method: "GET", url: "/api/bands?limit=999999" })).json().limit).toBe(150000);
});
it("rejette une recherche d'un seul caractère", async () => {
expect((await build().inject({ method: "GET", url: "/api/bands?q=a" })).statusCode).toBe(400);
});
it("rejette les patterns ILIKE pathologiques", async () => {
expect((await build().inject({ method: "GET", url: "/api/bands?q=%25%25%25" })).statusCode).toBe(400);
});
it("rejette plus de 100 pays", async () => {
const many = Array.from({ length: 101 }, (_, i) => `C${i}`).join(",");
const res = await build().inject({ method: "GET", url: `/api/bands?countries=${many}` });
expect(res.statusCode).toBe(400);
expect(res.json().error).toMatch(/Max 100/);
});
it("rejette plus de 50 statuts", async () => {
const many = Array.from({ length: 51 }, (_, i) => `s${i}`).join(",");
expect((await build().inject({ method: "GET", url: `/api/bands?status=${many}` })).statusCode).toBe(400);
});
it("passe la recherche en paramètre lié", async () => {
await build().inject({
method: "GET", url: "/api/bands?q=" + encodeURIComponent("'; DROP TABLE bands--"),
});
const call = pool.find("FROM bands");
expect(call.sql).not.toContain("DROP TABLE");
expect(call.values.some((v) => String(v).includes("DROP TABLE"))).toBe(true);
});
it("normalise les codes pays en majuscules", async () => {
await build().inject({ method: "GET", url: "/api/bands?countries=fr,de" });
expect(pool.find("FROM bands").values).toContainEqual(["FR", "DE"]);
});
});
describe("/api/clusters — validation géographique", () => {
it.each([
["bbox absente", "/api/clusters?zoom=5"],
["zoom absent", "/api/clusters?bbox=-5,40,10,55"],
["bbox à 3 valeurs", "/api/clusters?bbox=1,2,3&zoom=5"],
["bbox non numérique", "/api/clusters?bbox=a,b,c,d&zoom=5"],
["latitude > 90", "/api/clusters?bbox=-5,40,10,95&zoom=5"],
["longitude < -180", "/api/clusters?bbox=-181,40,10,55&zoom=5"],
["bbox inversée", "/api/clusters?bbox=10,40,-5,55&zoom=5"],
["zoom négatif", "/api/clusters?bbox=-5,40,10,55&zoom=-1"],
["zoom > 22", "/api/clusters?bbox=-5,40,10,55&zoom=23"],
])("répond 400 pour %s", async (_label, url) => {
expect((await build().inject({ method: "GET", url })).statusCode).toBe(400);
});
it("accepte une bbox valide", async () => {
const res = await build().inject({ method: "GET", url: "/api/clusters?bbox=-5,40,10,55&zoom=5" });
expect(res.statusCode).toBe(200);
});
// Le seuil de zoom 12 fait basculer entre agrégation en grille et points bruts.
it("agrège en clusters en dessous du zoom 12", async () => {
const res = await build().inject({ method: "GET", url: "/api/clusters?bbox=-5,40,10,55&zoom=11" });
expect(res.json().type).toBe("clusters");
expect(pool.find("grid_cells")).toBeDefined();
});
it("renvoie les groupes bruts à partir du zoom 12", async () => {
const res = await build().inject({ method: "GET", url: "/api/clusters?bbox=-5,40,10,55&zoom=12" });
expect(res.json().type).toBe("bands");
expect(pool.find("grid_cells")).toBeUndefined();
});
it("ne lit que les localisations réellement géocodées", async () => {
await build().inject({ method: "GET", url: "/api/clusters?bbox=-5,40,10,55&zoom=5" });
const sql = pool.find("grid_cells").sql;
expect(sql).toContain("bl.geom IS NOT NULL");
expect(sql).toContain("geocode_status IN ('done','country_only')");
});
it("utilise l'index spatial via ST_MakeEnvelope avec des paramètres liés", async () => {
await build().inject({ method: "GET", url: "/api/clusters?bbox=-5,40,10,55&zoom=5" });
const call = pool.find("grid_cells");
expect(call.sql).toContain("bl.geom && ST_MakeEnvelope($1, $2, $3, $4, 4326)");
expect(call.values.slice(0, 4)).toEqual([-5, 40, 10, 55]);
});
});
describe("/api/band/:ma_id", () => {
it("répond 400 pour un identifiant non numérique", async () => {
expect((await build().inject({ method: "GET", url: "/api/band/abc" })).statusCode).toBe(400);
});
it("répond 404 quand le groupe n'existe pas", async () => {
const res = await build([{ match: "FROM bands", result: rows() }])
.inject({ method: "GET", url: "/api/band/42" });
expect(res.statusCode).toBe(404);
});
it("accepte un ma_id au-delà de 2^31 (colonne BIGINT)", async () => {
const res = await build([{ match: "FROM bands", result: rows({ ma_id: 3500000000 }) }])
.inject({ method: "GET", url: "/api/band/3500000000" });
expect(res.statusCode).toBe(200);
expect(pool.find("FROM bands").values).toEqual([3500000000]);
});
});
describe("/admin/import — jeton Bearer", () => {
it.each([
["sans en-tête", undefined],
["jeton vide", "Bearer "],
["jeton faux de même longueur", `Bearer ${"0".repeat(TOKEN.length)}`],
["jeton faux plus court", "Bearer abc"],
["schéma Basic", "Basic " + Buffer.from("a:b").toString("base64")],
])("répond 401 %s", async (_label, authorization) => {
const res = await buildToken().inject({
method: "POST", url: "/admin/import",
headers: authorization ? { authorization } : {},
payload: { bands: [] },
});
expect(res.statusCode).toBe(401);
});
it("ne touche pas la base quand le jeton est invalide", async () => {
await buildToken().inject({
method: "POST", url: "/admin/import",
headers: { authorization: "Bearer faux" },
payload: { bands: [{ ma_id: 1 }] },
});
expect(tokenPool.calls).toHaveLength(0);
});
it("accepte le bon jeton", async () => {
const res = await buildToken([{ match: "INSERT INTO bands", result: rows() }]).inject({
method: "POST", url: "/admin/import",
headers: { authorization: `Bearer ${TOKEN}` },
payload: { bands: [{ ma_id: 1, name: "Mayhem" }] },
});
expect(res.statusCode).toBe(200);
expect(res.json().upserted).toBe(1);
});
// Un jeton multi-octets de même longueur en caractères mais pas en octets
// faisait lever timingSafeEqual → 500 au lieu d'un 401 propre.
it("répond 401 (pas 500) pour un jeton multi-octets", async () => {
const res = await buildToken().inject({
method: "POST", url: "/admin/import",
headers: { authorization: `Bearer ${"é".repeat(TOKEN.length)}` },
payload: { bands: [] },
});
expect(res.statusCode).toBe(401);
});
it("refuse un corps sans tableau bands", async () => {
for (const payload of [{}, { bands: "pas-un-tableau" }, { bands: null }]) {
const res = await buildToken().inject({
method: "POST", url: "/admin/import",
headers: { authorization: `Bearer ${TOKEN}` }, payload,
});
expect(res.statusCode).toBe(400);
}
});
it("refuse plus de 1000 groupes par lot", async () => {
const bands = Array.from({ length: 1001 }, (_, i) => ({ ma_id: i }));
const res = await buildToken().inject({
method: "POST", url: "/admin/import",
headers: { authorization: `Bearer ${TOKEN}` }, payload: { bands },
});
expect(res.statusCode).toBe(400);
});
it("ignore les entrées sans ma_id numérique sans planter", async () => {
const res = await buildToken([{ match: "INSERT INTO bands", result: rows() }]).inject({
method: "POST", url: "/admin/import",
headers: { authorization: `Bearer ${TOKEN}` },
payload: { bands: [{ ma_id: "abc" }, null, "chaine", { ma_id: 7 }] },
});
expect(res.statusCode).toBe(200);
expect(res.json().upserted).toBe(1);
});
// Cas nécessitant une fabrique différente : app sans jeton configuré.
it("quand aucun jeton n'est configuré, tout est refusé", async () => {
const noTokenApp = await createFullAppWith({ importToken: "", pool: null });
const res = await noTokenApp.inject({
method: "POST", url: "/admin/import",
headers: { authorization: "Bearer nimportequoi" }, payload: { bands: [] },
});
expect(res.statusCode).toBe(401);
await noTokenApp.close();
});
});
describe("/admin/auth/login", () => {
it("refuse un corps mal formé sans requête SQL", async () => {
const a = build([]);
for (const payload of [{}, { username: "a" }, { username: 1, password: 2 }]) {
expect((await a.inject({ method: "POST", url: "/admin/auth/login", payload })).statusCode).toBe(400);
}
expect(pool.calls).toHaveLength(0);
});
it("répond 429 quand le compte est verrouillé, sans vérifier le mot de passe", async () => {
const res = await build([
{ match: "admin_login_attempts", result: rows({ n: 99 }) },
{ match: "FROM admin_users", result: rows({ password_hash: "x" }) },
]).inject({
method: "POST", url: "/admin/auth/login",
payload: { username: "nico", password: "x" },
});
expect(res.statusCode).toBe(429);
expect(pool.find("FROM admin_users")).toBeUndefined();
});
it("enregistre la tentative échouée et ne pose pas de cookie", async () => {
const bcrypt = (await import("bcryptjs")).default;
// Hash à coût 4 : le DUMMY_HASH de production est à coût 12 (~330 ms).
const hash = bcrypt.hashSync("autre-chose", 4);
const res = await build([
{ match: "SELECT count(*)::int AS n", result: rows({ n: 0 }) },
{ match: "FROM admin_users", result: rows({ password_hash: hash }) },
{ match: "INSERT INTO admin_login_attempts", result: rows() },
]).inject({
method: "POST", url: "/admin/auth/login",
payload: { username: "nico", password: "mauvais" },
});
expect(res.statusCode).toBe(401);
expect(res.headers["set-cookie"]).toBeUndefined();
expect(pool.find("INSERT INTO admin_login_attempts").values[2]).toBe(false);
});
it("le cookie de session est httpOnly, secure, sameSite=strict", async () => {
const bcrypt = (await import("bcryptjs")).default;
const hash = bcrypt.hashSync("bon", 4);
const res = await build([
{ match: "SELECT count(*)::int AS n", result: rows({ n: 0 }) },
{ match: "FROM admin_users", result: rows({ password_hash: hash }) },
{ match: "INSERT INTO admin_login_attempts", result: rows() },
{ match: "UPDATE admin_users", result: rows() },
]).inject({
method: "POST", url: "/admin/auth/login",
payload: { username: "nico", password: "bon" },
});
expect(res.statusCode).toBe(200);
const cookie = String(res.headers["set-cookie"]);
expect(cookie).toMatch(/HttpOnly/i);
expect(cookie).toMatch(/Secure/i);
expect(cookie).toMatch(/SameSite=Strict/i);
// Le jeton ne doit jamais apparaître dans le corps de la réponse
expect(res.json()).toEqual({ ok: true, username: "nico" });
});
it("tronque un username exagérément long avant la requête", async () => {
const bcrypt = (await import("bcryptjs")).default;
const hash = bcrypt.hashSync("x", 4);
await build([
{ match: "SELECT count(*)::int AS n", result: rows({ n: 0 }) },
{ match: "FROM admin_users", result: rows({ password_hash: hash }) },
{ match: "INSERT INTO admin_login_attempts", result: rows() },
]).inject({
method: "POST", url: "/admin/auth/login",
payload: { username: "a".repeat(5000), password: "x" },
});
expect(pool.find("FROM admin_users").values[0]).toHaveLength(100);
});
});
describe("gestionnaire d'erreurs global", () => {
// Régression : setErrorHandler écrasait TOUS les statuts en 500, y compris
// les 4xx légitimes (payload trop gros, JSON invalide, rate-limit).
it("préserve un 400 sur JSON invalide", async () => {
const res = await build([]).inject({
method: "POST", url: "/admin/auth/login",
headers: { "content-type": "application/json" },
payload: "{ceci n'est pas du json",
});
expect(res.statusCode).toBe(400);
});
it("renvoie 500 générique pour une erreur inattendue", async () => {
const res = await build([{ match: "FROM bands", throws: new Error("boom") }])
.inject({ method: "GET", url: "/api/stats" });
expect(res.statusCode).toBe(500);
expect(res.json().error).not.toContain("boom");
});
});
describe("sans base de données configurée", () => {
it("les routes de données répondent 500 mais le service reste debout", async () => {
const noDbApp = await createFullAppWith({ pool: null });
expect((await noDbApp.inject({ method: "GET", url: "/api/stats" })).statusCode).toBe(500);
expect((await noDbApp.inject({ method: "GET", url: "/api/health" })).statusCode).toBe(200);
await noDbApp.close();
});
});

View file

@ -0,0 +1,393 @@
import { describe, it, expect, beforeAll, afterAll } from "vitest";
process.env.ADMIN_JWT_SECRET = "secret-de-test-suffisamment-long-pour-passer-32";
const { signAdminSession, ADMIN_COOKIE_NAME } = await import("../src/adminAuth.js");
const { rows } = await import("./helpers/fakePool.js");
const { createFullApp } = await import("./helpers/testApp.js");
const TOKEN_FOR_APP = "jeton-de-test";
// Deux apps seulement pour tout le fichier : une par jeu d'options de fabrique.
// Chaque test se contente de reprogrammer le pool associé.
let app, pool, tokenApp, tokenPool;
beforeAll(async () => {
({ app, pool } = await createFullApp());
({ app: tokenApp, pool: tokenPool } = await createFullApp({ importToken: TOKEN_FOR_APP }));
});
afterAll(async () => { await app.close(); await tokenApp.close(); });
/** Reprogramme le pool partagé et renvoie l'app publique. */
function build(handlers) {
pool.reset(handlers ?? []);
return app;
}
/** Idem pour l'app configurée avec un jeton d'import. */
function buildToken(handlers) {
tokenPool.reset(handlers ?? []);
return tokenApp;
}
const anyPool = (extra = []) =>
[...extra, { match: /./, result: rows({ n: 1, total: 1, value: "x", count: 1 }) }];
describe("routes d'agrégation publiques", () => {
const ROUTES = ["/api/db", "/api/stats", "/api/countries", "/api/statuses", "/api/facets"];
it.each(ROUTES)("%s répond 200 avec ok:true", async (url) => {
const app = build(anyPool());
const res = await app.inject({ method: "GET", url });
expect(res.statusCode).toBe(200);
expect(res.json().ok).toBe(true);
});
it.each(ROUTES)("%s répond 500 sans divulguer l'erreur si la base tombe", async (url) => {
const handlers = ([{ match: /./, throws: new Error("FATAL: role bm does not exist") }]);
const app = build(handlers);
const res = await app.inject({ method: "GET", url });
expect(res.statusCode).toBe(500);
expect(res.body).not.toMatch(/role bm|FATAL/);
});
it.each(ROUTES)("%s est accessible sans authentification (données publiques)", async (url) => {
const app = build(anyPool());
expect((await app.inject({ method: "GET", url })).statusCode).not.toBe(401);
});
});
describe("/api/stats", () => {
it("expose les compteurs attendus par le frontend", async () => {
const handlers = ([
{ match: "FROM bands", result: rows({ total: 100, geocoded: 80, no_location: 20, enriched: 60 }) },
]);
const app = build(handlers);
const body = (await app.inject({ method: "GET", url: "/api/stats" })).json();
expect(body).toMatchObject({ ok: true, total: 100, geocoded: 80, no_location: 20, enriched: 60 });
});
// geocoded compte les GROUPES ayant un point principal (bands.geom), pas les
// localisations. La carte, elle, clusterise band_locations : les deux chiffres
// n'ont pas la même unité et ne doivent pas être comparés naïvement.
it("compte les groupes via bands.geom, pas les localisations", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: "/api/stats" });
const sql = pool.find("FROM bands").sql;
expect(sql).toContain("geom IS NOT NULL");
expect(sql).not.toContain("band_locations");
});
});
describe("/api/facets", () => {
it("ne propose que des facettes portant sur des groupes géocodés", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: "/api/facets" });
// Sinon l'utilisateur peut cocher une facette qui ne fera apparaître
// aucun point sur la carte.
for (const call of pool.calls) {
expect(call.sql).toContain("geom IS NOT NULL");
}
});
it("borne la liste des genres", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: "/api/facets" });
expect(pool.find("genre as value").sql).toContain("LIMIT 500");
});
it("renvoie une plage d'années même quand la base est vide", async () => {
const handlers = ([{ match: /./, result: rows() }]);
const app = build(handlers);
const body = (await app.inject({ method: "GET", url: "/api/facets" })).json();
expect(body.year_range).toEqual({ min_year: null, max_year: null });
});
it("écarte les années aberrantes de la plage", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: "/api/facets" });
const sql = pool.find("MIN(formed_year)").sql;
expect(sql).toContain("formed_year >= 1900");
expect(sql).toContain("extract(year from now())");
});
});
describe("/api/bands — filtres", () => {
it("filtre sur la présence de coordonnées", async () => {
const mk = async (qs) => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: `/api/bands${qs}` });
return pool.find("FROM bands").sql;
};
expect(await mk("?geocoded=1")).toContain("geom IS NOT NULL");
expect(await mk("?geocoded=0")).toContain("geom IS NULL");
expect(await mk("")).not.toContain("geom IS");
});
it("only_black=1 filtre sur le genre", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: "/api/bands?only_black=1" });
expect(pool.find("FROM bands").sql).toContain("genre ILIKE '%black%'");
});
it("combine plusieurs filtres avec AND", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: "/api/bands?countries=NO&geocoded=1&status=Active" });
const sql = pool.find("FROM bands").sql;
expect(sql).toContain(" AND ");
expect(sql.match(/ AND /g).length).toBeGreaterThanOrEqual(2);
});
it("les statuts vides sont normalisés en 'Unknown'", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: "/api/bands?status=Unknown" });
expect(pool.find("FROM bands").sql).toContain("COALESCE(NULLIF(trim(status), ''), 'Unknown')");
});
it("un tri stable est imposé (pagination cohérente)", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: "/api/bands" });
// Sans ORDER BY déterministe, deux pages successives peuvent renvoyer
// deux fois le même groupe et en omettre un autre.
expect(pool.find("FROM bands").sql).toContain("ORDER BY ma_id ASC");
});
});
describe("/api/clusters — filtres", () => {
const base = "/api/clusters?bbox=-5,40,10,55&zoom=5";
it("applique le filtre de genre en paramètre lié", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: `${base}&genre=black` });
const call = pool.find("grid_cells");
expect(call.sql).toContain("b.genre ILIKE");
expect(call.values).toContain("%black%");
});
it("rejette un genre trop court ou pathologique", async () => {
const app = build(anyPool());
expect((await app.inject({ method: "GET", url: `${base}&genre=a` })).statusCode).toBe(400);
expect((await app.inject({ method: "GET", url: `${base}&genre=%25%25%25` })).statusCode).toBe(400);
});
it.each([
["year_min=1799", 400], ["year_min=2101", 400], ["year_min=abc", 400],
["year_max=1799", 400], ["year_max=abc", 400],
["year_min=1980&year_max=1990", 200],
])("%s → %i", async (qs, expected) => {
const app = build(anyPool());
expect((await app.inject({ method: "GET", url: `${base}&${qs}` })).statusCode).toBe(expected);
});
it("refuse plus de 100 pays et plus de 50 statuts", async () => {
const app = build(anyPool());
const c = Array.from({ length: 101 }, (_, i) => `C${i}`).join(",");
const s = Array.from({ length: 51 }, (_, i) => `s${i}`).join(",");
expect((await app.inject({ method: "GET", url: `${base}&countries=${c}` })).statusCode).toBe(400);
expect((await app.inject({ method: "GET", url: `${base}&status=${s}` })).statusCode).toBe(400);
});
it("la taille de cellule décroît quand le zoom augmente", async () => {
const cell = async (zoom) => {
const app = build(anyPool());
const body = (await app.inject({ method: "GET", url: `/api/clusters?bbox=-5,40,10,55&zoom=${zoom}` })).json();
return body.cell_size;
};
expect(await cell(3)).toBeGreaterThan(await cell(8));
});
it("la taille de cellule a un plancher (pas de division infinie)", async () => {
const app = build(anyPool());
const body = (await app.inject({ method: "GET", url: "/api/clusters?bbox=-5,40,10,55&zoom=11" })).json();
expect(body.cell_size).toBeGreaterThanOrEqual(0.01);
});
it("borne le nombre de résultats", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: base });
expect(pool.find("grid_cells").sql).toContain("LIMIT 1000");
});
it("borne aussi le mode points bruts", async () => {
const handlers = anyPool();
const app = build(handlers);
await app.inject({ method: "GET", url: "/api/clusters?bbox=-5,40,10,55&zoom=14" });
expect(pool.find("FROM band_locations bl").sql).toContain("LIMIT 2000");
});
});
describe("/admin/enrich/next", () => {
const TOKEN = TOKEN_FOR_APP;
const withToken = { authorization: `Bearer ${TOKEN}` };
it("exige le jeton Bearer", async () => {
const handlers = ([]);
const app = buildToken(handlers);
expect((await app.inject({ method: "GET", url: "/admin/enrich/next" })).statusCode).toBe(401);
expect(tokenPool.calls).toHaveLength(0);
});
it("ne renvoie que les groupes ayant une URL", async () => {
const handlers = ([
{ match: "FROM bands", result: rows({ ma_id: 1, url: "http://x" }, { ma_id: 2, url: null }) },
]);
const app = buildToken(handlers);
const body = (await app.inject({ method: "GET", url: "/admin/enrich/next", headers: withToken })).json();
expect(body.count).toBe(1);
expect(body.items).toEqual([{ ma_id: 1, url: "http://x" }]);
});
it("borne la limite entre 1 et 500", async () => {
const lim = async (q) => {
const handlers = ([{ match: "FROM bands", result: rows() }]);
const app = buildToken(handlers);
await app.inject({ method: "GET", url: `/admin/enrich/next?limit=${q}`, headers: withToken });
return tokenPool.find("FROM bands").values.at(-1);
};
expect(await lim(9999)).toBe(500);
expect(await lim(0)).toBe(1);
expect(await lim(-5)).toBe(1);
expect(await lim("abc")).toBe(50); // valeur par défaut, pas NaN
expect(await lim(25)).toBe(25);
});
it("filtre par pays, normalisé en majuscules et lié", async () => {
const handlers = ([{ match: "FROM bands", result: rows() }]);
const app = buildToken(handlers);
await app.inject({ method: "GET", url: "/admin/enrich/next?country=no", headers: withToken });
const call = tokenPool.find("FROM bands");
expect(call.sql).toContain("country = $1");
expect(call.values[0]).toBe("NO");
});
it("ne sélectionne que les groupes jamais enrichis", async () => {
const handlers = ([{ match: "FROM bands", result: rows() }]);
const app = buildToken(handlers);
await app.inject({ method: "GET", url: "/admin/enrich/next", headers: withToken });
expect(tokenPool.find("FROM bands").sql).toContain("(data->'band_page') IS NULL");
});
});
describe("/admin/auth/me et logout", () => {
it("me répond 401 sans cookie", async () => {
const app = build(anyPool());
expect((await app.inject({ method: "GET", url: "/admin/auth/me" })).statusCode).toBe(401);
});
it("me renvoie le username de la session", async () => {
const app = build(anyPool());
const res = await app.inject({
method: "GET", url: "/admin/auth/me",
headers: { cookie: `${ADMIN_COOKIE_NAME}=${signAdminSession("nico")}` },
});
expect(res.json()).toEqual({ ok: true, username: "nico" });
});
it("logout efface le cookie", async () => {
const app = build(anyPool());
const res = await app.inject({ method: "POST", url: "/admin/auth/logout" });
expect(res.statusCode).toBe(200);
const cookie = String(res.headers["set-cookie"]);
expect(cookie).toContain(ADMIN_COOKIE_NAME);
expect(cookie).toMatch(/Expires=Thu, 01 Jan 1970|Max-Age=0/i);
});
it("logout ne nécessite pas de session valide (idempotent)", async () => {
const app = build(anyPool());
expect((await app.inject({ method: "POST", url: "/admin/auth/logout" })).statusCode).toBe(200);
});
});
describe("/admin/import — upsert", () => {
const TOKEN = TOKEN_FOR_APP;
const withToken = { authorization: `Bearer ${TOKEN}` };
const importPool = () => ([{ match: "INSERT INTO bands", result: rows() }]);
it("préserve les valeurs existantes quand le champ est absent (COALESCE)", async () => {
const handlers = importPool();
const app = buildToken(handlers);
await app.inject({
method: "POST", url: "/admin/import", headers: withToken,
payload: { bands: [{ ma_id: 1, name: "Mayhem" }] },
});
const sql = tokenPool.find("INSERT INTO bands").sql;
expect(sql).toContain("ON CONFLICT (ma_id) DO UPDATE");
expect(sql).toContain("COALESCE(EXCLUDED.name, bands.name)");
});
it("accepte `location` comme alias de `location_text`", async () => {
const handlers = importPool();
const app = buildToken(handlers);
await app.inject({
method: "POST", url: "/admin/import", headers: withToken,
payload: { bands: [{ ma_id: 1, location: "Oslo" }] },
});
expect(tokenPool.find("INSERT INTO bands").values[5]).toBe("Oslo");
});
it("marque enriched quand un objet data est fourni", async () => {
const handlers = importPool();
const app = buildToken(handlers);
await app.inject({
method: "POST", url: "/admin/import", headers: withToken,
payload: { bands: [{ ma_id: 1, data: { band_page: "..." } }] },
});
expect(tokenPool.find("INSERT INTO bands").values[7]).toBe(true);
});
it("un data non-objet n'active pas enriched", async () => {
const handlers = importPool();
const app = buildToken(handlers);
await app.inject({
method: "POST", url: "/admin/import", headers: withToken,
payload: { bands: [{ ma_id: 1, data: "pas-un-objet" }] },
});
const vals = tokenPool.find("INSERT INTO bands").values;
expect(vals[6]).toBeNull();
expect(vals[7]).toBeNull();
});
it("tous les champs passent en paramètres liés", async () => {
const handlers = importPool();
const app = buildToken(handlers);
await app.inject({
method: "POST", url: "/admin/import", headers: withToken,
payload: { bands: [{ ma_id: 1, name: "'; DROP TABLE bands--" }] },
});
const call = tokenPool.find("INSERT INTO bands");
expect(call.sql).not.toContain("DROP TABLE");
expect(call.values).toContain("'; DROP TABLE bands--");
});
it("compte exactement les lignes traitées", async () => {
const handlers = importPool();
const app = buildToken(handlers);
const res = await app.inject({
method: "POST", url: "/admin/import", headers: withToken,
payload: { bands: [{ ma_id: 1 }, { ma_id: 2 }, { ma_id: 3 }] },
});
expect(res.json().upserted).toBe(3);
expect(tokenPool.findAll("INSERT INTO bands")).toHaveLength(3);
});
it("accepte un lot de exactement 1000 groupes", async () => {
const app = buildToken(importPool());
const bands = Array.from({ length: 1000 }, (_, i) => ({ ma_id: i + 1 }));
const res = await app.inject({
method: "POST", url: "/admin/import", headers: withToken, payload: { bands },
});
expect(res.statusCode).toBe(200);
});
});

View file

@ -0,0 +1,126 @@
import { describe, it, expect } from "vitest";
process.env.ADMIN_JWT_SECRET = "secret-de-test-suffisamment-long-pour-passer-32";
const { createFullAppWith } = await import("./helpers/testApp.js");
const { makeFakePool, rows } = await import("./helpers/fakePool.js");
/**
* Les autres fichiers de test partagent une app avec des plafonds de débit
* neutralisés (sinon l'état du limiteur fuiterait d'un cas à l'autre). C'EST
* ICI, et seulement ici, que les vraies valeurs de production sont vérifiées
* sinon la paramétrisation introduite pour la performance créerait un angle
* mort sur une protection de sécurité.
*/
const TOKEN = "jeton-de-test";
async function appWithRealLimits(overrides = {}) {
return createFullAppWith({
pool: makeFakePool([{ match: /./, result: rows({ total: 0 }) }]),
importToken: TOKEN,
...overrides,
});
}
describe("plafond global", () => {
it("répond 429 au-delà du plafond", async () => {
const app = await appWithRealLimits({ globalRateLimitMax: 3 });
const codes = [];
for (let i = 0; i < 5; i++) {
codes.push((await app.inject({ method: "GET", url: "/api/health" })).statusCode);
}
expect(codes).toEqual([200, 200, 200, 429, 429]);
await app.close();
});
it("le 429 traverse le gestionnaire d'erreurs sans devenir un 500", async () => {
// Régression : setErrorHandler écrasait tous les statuts en 500, ce qui
// transformait une limitation de débit en fausse panne serveur.
const app = await appWithRealLimits({ globalRateLimitMax: 1 });
await app.inject({ method: "GET", url: "/api/health" });
const res = await app.inject({ method: "GET", url: "/api/health" });
expect(res.statusCode).toBe(429);
expect(res.json().ok).toBe(false);
await app.close();
});
it("la valeur de production par défaut est 1000/minute", async () => {
const app = await appWithRealLimits();
const res = await app.inject({ method: "GET", url: "/api/health" });
expect(res.headers["x-ratelimit-limit"]).toBe("1000");
await app.close();
});
});
describe("plafond de /admin/import", () => {
it("la valeur de production par défaut est 10/minute", async () => {
const app = await appWithRealLimits();
const res = await app.inject({
method: "POST", url: "/admin/import",
headers: { authorization: `Bearer ${TOKEN}` }, payload: { bands: [] },
});
expect(res.headers["x-ratelimit-limit"]).toBe("10");
await app.close();
});
it("limite par jeton, pas globalement", async () => {
const app = await appWithRealLimits({ adminRateLimitMax: 2 });
const call = (token) =>
app.inject({
method: "POST", url: "/admin/import",
headers: { authorization: `Bearer ${token}` }, payload: { bands: [] },
});
await call(TOKEN);
await call(TOKEN);
// Le 3e appel avec CE jeton est bloqué…
expect((await call(TOKEN)).statusCode).toBe(429);
// …mais un autre jeton dispose de son propre compteur (il sera rejeté en
// 401, pas en 429 : c'est bien l'authentification qui tranche, pas le débit).
expect((await call("un-autre-jeton-de-la-meme-taille")).statusCode).toBe(401);
await app.close();
});
});
describe("plafond de /admin/auth/login", () => {
it("la valeur de production par défaut est 10/minute", async () => {
const app = await appWithRealLimits();
const res = await app.inject({
method: "POST", url: "/admin/auth/login", payload: { username: "", password: "" },
});
expect(res.headers["x-ratelimit-limit"]).toBe("10");
await app.close();
});
// Le verrouillage de compte (5 échecs / 15 min, en base) et le rate-limit HTTP
// sont deux protections distinctes : celle-ci s'applique même à des requêtes
// malformées qui n'atteignent jamais la base.
it("limite le bruteforce même sur des corps invalides", async () => {
const app = await appWithRealLimits({ authRateLimitMax: 2 });
const call = () =>
app.inject({ method: "POST", url: "/admin/auth/login", payload: {} });
expect((await call()).statusCode).toBe(400);
expect((await call()).statusCode).toBe(400);
expect((await call()).statusCode).toBe(429);
await app.close();
});
it("limite par IP", async () => {
const app = await appWithRealLimits({ authRateLimitMax: 1 });
const call = (ip) =>
app.inject({
method: "POST", url: "/admin/auth/login",
headers: { "x-forwarded-for": ip },
payload: {},
});
expect((await call("1.2.3.4")).statusCode).toBe(400);
expect((await call("1.2.3.4")).statusCode).toBe(429);
// Une autre IP garde son propre quota (trustProxy est actif : l'app est
// derrière Traefik, l'IP réelle vient de X-Forwarded-For).
expect((await call("5.6.7.8")).statusCode).toBe(400);
await app.close();
});
});

View file

@ -0,0 +1,220 @@
import { describe, it, expect } from "vitest";
import {
ValidationError,
sanitizeSearchString,
parseLimitOffset,
pagination,
parseBbox,
parseZoom,
cellSizeForZoom,
parseCsvList,
parseYear,
parseLat,
parseLon,
parseMaId,
} from "../src/validate.js";
describe("sanitizeSearchString", () => {
it("trim et accepte une recherche normale", () => {
expect(sanitizeSearchString(" mayhem ")).toBe("mayhem");
});
it.each([
["trop courte", "a"],
["vide", " "],
["trop longue", "x".repeat(101)],
])("rejette une chaîne %s", (_label, input) => {
expect(() => sanitizeSearchString(input)).toThrow(ValidationError);
});
it("accepte pile 2 et 100 caractères (bornes incluses)", () => {
expect(sanitizeSearchString("ab")).toBe("ab");
expect(sanitizeSearchString("y".repeat(100))).toHaveLength(100);
});
it("rejette les caractères de contrôle", () => {
expect(() => sanitizeSearchString("may\x00hem")).toThrow(/caractères invalides/);
expect(() => sanitizeSearchString("may\x7Fhem")).toThrow(/caractères invalides/);
});
it("laisse passer tab et newline (hors plage de contrôle bloquée)", () => {
expect(sanitizeSearchString("a\tb")).toBe("a\tb");
});
it("rejette les patterns ILIKE pathologiques", () => {
expect(() => sanitizeSearchString("%%%")).toThrow(/patterns invalides/);
expect(() => sanitizeSearchString("_".repeat(10))).toThrow(/patterns invalides/);
expect(() => sanitizeSearchString("%".repeat(10))).toThrow(/patterns invalides/);
});
it("autorise un % isolé (recherche partielle légitime)", () => {
expect(sanitizeSearchString("100%")).toBe("100%");
});
it("utilise le nom de champ dans le message", () => {
expect(() => sanitizeSearchString("a", "Genre")).toThrow(/^Genre /);
});
});
describe("parseLimitOffset", () => {
it("applique les valeurs par défaut", () => {
expect(parseLimitOffset(undefined, undefined)).toEqual({ limit: 1000, offset: 0 });
expect(parseLimitOffset("", "")).toEqual({ limit: 1000, offset: 0 });
});
it("borne le limit au maximum", () => {
expect(parseLimitOffset("999999", "0", { maxLimit: 150000 }).limit).toBe(150000);
});
it("tronque les valeurs fractionnaires", () => {
expect(parseLimitOffset("10.9", "5.9")).toEqual({ limit: 10, offset: 5 });
});
// Régression : `Number("abc")` donnait NaN, propagé jusqu'au LIMIT $n
// → erreur pg → 500 au lieu d'un 400 explicite.
it("rejette un limit non numérique au lieu de propager NaN", () => {
expect(() => parseLimitOffset("abc", "0")).toThrow(ValidationError);
expect(() => parseLimitOffset("0", "0")).toThrow(/limit invalide/);
expect(() => parseLimitOffset("-5", "0")).toThrow(/limit invalide/);
});
it("rejette un offset non numérique ou négatif", () => {
expect(() => parseLimitOffset("10", "abc")).toThrow(/offset invalide/);
expect(() => parseLimitOffset("10", "-1")).toThrow(/offset invalide/);
});
it("rejette un offset au-delà du million", () => {
expect(() => parseLimitOffset("10", "1000001")).toThrow(/Offset trop grand/);
expect(parseLimitOffset("10", "1000000").offset).toBe(1000000);
});
});
describe("pagination", () => {
it("valeurs par défaut", () => {
expect(pagination({})).toEqual({ page: 1, pageSize: 50, offset: 0 });
});
it("calcule l'offset", () => {
expect(pagination({ page: "3", pageSize: "20" })).toEqual({ page: 3, pageSize: 20, offset: 40 });
});
it("borne pageSize à 200 et page à 1 minimum", () => {
expect(pagination({ pageSize: "10000" }).pageSize).toBe(200);
expect(pagination({ page: "-4" }).page).toBe(1);
expect(pagination({ page: "abc" }).page).toBe(1);
});
});
describe("parseBbox", () => {
it("parse une bbox valide", () => {
expect(parseBbox("-5,40,10,55")).toEqual({ minLon: -5, minLat: 40, maxLon: 10, maxLat: 55 });
});
it.each([
["absente", ""],
["3 valeurs", "1,2,3"],
["5 valeurs", "1,2,3,4,5"],
["non numérique", "a,b,c,d"],
["hors bornes lat", "-5,40,10,95"],
["hors bornes lon", "-181,40,10,55"],
["minLon >= maxLon", "10,40,10,55"],
["minLat >= maxLat", "-5,55,10,40"],
])("rejette une bbox %s", (_label, input) => {
expect(() => parseBbox(input)).toThrow(ValidationError);
});
});
describe("parseZoom / cellSizeForZoom", () => {
it("accepte les bornes 0 et 22", () => {
expect(parseZoom("0")).toBe(0);
expect(parseZoom("22")).toBe(22);
});
it("rejette hors bornes et non numérique", () => {
expect(() => parseZoom("-1")).toThrow(/between 0 and 22/);
expect(() => parseZoom("23")).toThrow(/between 0 and 22/);
expect(() => parseZoom("abc")).toThrow(ValidationError);
});
it("la taille de cellule décroît avec le zoom et a un plancher", () => {
expect(cellSizeForZoom(0)).toBe(180);
expect(cellSizeForZoom(4)).toBeCloseTo(11.25);
expect(cellSizeForZoom(22)).toBe(0.01);
expect(cellSizeForZoom(11)).toBeGreaterThan(cellSizeForZoom(12));
});
});
describe("parseCsvList", () => {
it("découpe, trim et filtre les vides", () => {
expect(parseCsvList("fr, de ,, be")).toEqual(["fr", "de", "be"]);
});
it("applique la transformation", () => {
expect(parseCsvList("fr,de", { transform: (s) => s.toUpperCase() })).toEqual(["FR", "DE"]);
});
it("rejette au-delà du maximum", () => {
const many = Array.from({ length: 101 }, (_, i) => `c${i}`).join(",");
expect(() => parseCsvList(many, { max: 100, label: "pays" })).toThrow(/Max 100 pays/);
expect(parseCsvList(many, { max: 101, label: "pays" })).toHaveLength(101);
});
});
describe("parseYear", () => {
it("renvoie null pour une entrée vide", () => {
expect(parseYear("")).toBeNull();
expect(parseYear(null)).toBeNull();
expect(parseYear(undefined)).toBeNull();
});
it("accepte les bornes 1800 et 2100", () => {
expect(parseYear("1800")).toBe(1800);
expect(parseYear("2100")).toBe(2100);
});
it("rejette hors bornes et non numérique", () => {
expect(() => parseYear("1799")).toThrow(ValidationError);
expect(() => parseYear("2101")).toThrow(ValidationError);
expect(() => parseYear("abc", "year_min")).toThrow(/^year_min invalide/);
});
});
describe("parseLat / parseLon", () => {
it("accepte les bornes", () => {
expect(parseLat(-90)).toBe(-90);
expect(parseLat(90)).toBe(90);
expect(parseLon(-180)).toBe(-180);
expect(parseLon(180)).toBe(180);
});
it("null efface la valeur", () => {
expect(parseLat(null)).toBeNull();
expect(parseLon("")).toBeNull();
});
it("rejette hors bornes", () => {
expect(() => parseLat(90.1)).toThrow(/lat invalide/);
expect(() => parseLon(-180.1)).toThrow(/lon invalide/);
expect(() => parseLat("nord")).toThrow(/lat invalide/);
});
// 0 est une coordonnée valide (golfe de Guinée) : ne doit pas être traité
// comme une valeur vide par un test de falsiness.
it("accepte 0", () => {
expect(parseLat(0)).toBe(0);
expect(parseLon(0)).toBe(0);
});
});
describe("parseMaId", () => {
it("accepte un entier positif, y compris au-delà de 2^31", () => {
expect(parseMaId("3500000000")).toBe(3500000000);
expect(parseMaId(0)).toBe(0);
});
it("rejette négatif, décimal et non numérique", () => {
expect(() => parseMaId("-1")).toThrow(/bad ma_id/);
expect(() => parseMaId("1.5")).toThrow(/bad ma_id/);
expect(() => parseMaId("abc")).toThrow(/bad ma_id/);
});
});

View file

@ -3,9 +3,10 @@
""" """
import json import json
import logging import logging
import time
from contextlib import contextmanager from contextlib import contextmanager
from datetime import datetime, timezone from datetime import UTC, datetime
from typing import Any, Dict, List, Optional from typing import Any
import psycopg2 import psycopg2
import psycopg2.extras import psycopg2.extras
@ -29,14 +30,14 @@ def get_conn():
def now_utc() -> datetime: def now_utc() -> datetime:
return datetime.now(timezone.utc) return datetime.now(UTC)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Bands # Bands
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def upsert_bands(bands: List[Dict[str, Any]]) -> Dict[str, int]: def upsert_bands(bands: list[dict[str, Any]]) -> dict[str, int]:
""" """
Upsert en bulk. Préserve les champs existants si la nouvelle valeur est None. Upsert en bulk. Préserve les champs existants si la nouvelle valeur est None.
Retourne {"inserted": N, "updated": N}. Retourne {"inserted": N, "updated": N}.
@ -99,7 +100,7 @@ def upsert_bands(bands: List[Dict[str, Any]]) -> Dict[str, int]:
return {"inserted": inserted, "updated": updated} return {"inserted": inserted, "updated": updated}
def upsert_band_enriched(ma_id: int, data: Dict[str, Any], html_hash: str) -> bool: def upsert_band_enriched(ma_id: int, data: dict[str, Any], html_hash: str) -> bool:
""" """
Met à jour les champs d'enrichissement d'un band. Met à jour les champs d'enrichissement d'un band.
Respecte locked_fields : les champs édités manuellement dans l'admin ne sont pas Respecte locked_fields : les champs édités manuellement dans l'admin ne sont pas
@ -107,7 +108,7 @@ def upsert_band_enriched(ma_id: int, data: Dict[str, Any], html_hash: str) -> bo
stockée dans crawler_pending pour résolution manuelle. stockée dans crawler_pending pour résolution manuelle.
""" """
ts = now_utc() ts = now_utc()
new_vals: Dict[str, Any] = { new_vals: dict[str, Any] = {
"status": data.get("status"), "status": data.get("status"),
"genre": data.get("genre"), "genre": data.get("genre"),
"themes": data.get("themes"), "themes": data.get("themes"),
@ -157,7 +158,7 @@ def upsert_band_enriched(ma_id: int, data: Dict[str, Any], html_hash: str) -> bo
return cur.rowcount > 0 return cur.rowcount > 0
def get_bands_to_enrich(country: Optional[str] = None, limit: int = 50) -> List[Dict]: def get_bands_to_enrich(country: str | None = None, limit: int = 50) -> list[dict]:
""" """
File de priorité pour l'enrichissement : File de priorité pour l'enrichissement :
1. Nouveaux bands (band_page absent) jamais enrichis 1. Nouveaux bands (band_page absent) jamais enrichis
@ -203,7 +204,7 @@ def get_bands_to_enrich(country: Optional[str] = None, limit: int = 50) -> List[
# Logs (visibles dans le dashboard admin) # Logs (visibles dans le dashboard admin)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def log_event(level: str, message: str, run_id: Optional[int] = None, ma_id: Optional[int] = None): def log_event(level: str, message: str, run_id: int | None = None, ma_id: int | None = None):
"""Écrit une ligne de log en DB pour le dashboard admin (n'interrompt jamais le crawl).""" """Écrit une ligne de log en DB pour le dashboard admin (n'interrompt jamais le crawl)."""
try: try:
with get_conn() as conn: with get_conn() as conn:
@ -220,7 +221,7 @@ def log_event(level: str, message: str, run_id: Optional[int] = None, ma_id: Opt
# Crawl run tracking # Crawl run tracking
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def update_crawl_run_progress(run_id: int, stats: Dict[str, int]): def update_crawl_run_progress(run_id: int, stats: dict[str, int]):
"""Mise à jour des compteurs d'un run en cours (progression live).""" """Mise à jour des compteurs d'un run en cours (progression live)."""
try: try:
with get_conn() as conn: with get_conn() as conn:
@ -236,7 +237,7 @@ def update_crawl_run_progress(run_id: int, stats: Dict[str, int]):
log.warning(f"[db] update_crawl_run_progress failed: {e}") log.warning(f"[db] update_crawl_run_progress failed: {e}")
def start_crawl_run(run_type: str, countries: Optional[List[str]] = None) -> int: def start_crawl_run(run_type: str, countries: list[str] | None = None) -> int:
sql = "INSERT INTO crawl_run (run_type, countries) VALUES (%s, %s) RETURNING id" sql = "INSERT INTO crawl_run (run_type, countries) VALUES (%s, %s) RETURNING id"
with get_conn() as conn: with get_conn() as conn:
with conn.cursor() as cur: with conn.cursor() as cur:
@ -244,8 +245,62 @@ def start_crawl_run(run_type: str, countries: Optional[List[str]] = None) -> int
return cur.fetchone()[0] return cur.fetchone()[0]
def finish_crawl_run(run_id: int, stats: Dict[str, int], error: Optional[str] = None): class RunCancelled(Exception):
status = "error" if error else "done" """Levée dans les boucles de crawl quand un admin demande l'annulation.
Volontairement distincte des erreurs : un run annulé n'est pas un échec, il
ne doit ni être marqué 'error' ni polluer les alertes.
"""
# Le drapeau d'annulation est relu au plus une fois toutes les
# _CANCEL_POLL_SECONDS secondes : les boucles de crawl appellent le check à
# chaque groupe, on ne veut pas une requête SQL par groupe.
_CANCEL_POLL_SECONDS = 5.0
_cancel_cache: dict[int, tuple[float, bool]] = {}
def is_cancel_requested(run_id: int, force: bool = False) -> bool:
"""Un admin a-t-il demandé l'arrêt de ce run ?
En cas d'erreur DB on renvoie False : une base momentanément injoignable ne
doit pas interrompre un crawl en cours.
"""
now = time.monotonic()
cached = _cancel_cache.get(run_id)
if not force and cached and (now - cached[0]) < _CANCEL_POLL_SECONDS:
return cached[1]
try:
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT cancel_requested FROM crawl_run WHERE id = %s", (run_id,)
)
row = cur.fetchone()
requested = bool(row and row[0])
except Exception as e:
log.warning(f"[db] is_cancel_requested failed: {e}")
return cached[1] if cached else False
_cancel_cache[run_id] = (now, requested)
return requested
def raise_if_cancelled(run_id: int):
"""Point de contrôle à placer dans les boucles longues."""
if is_cancel_requested(run_id):
raise RunCancelled(f"run #{run_id} annulé par un administrateur")
def finish_crawl_run(
run_id: int,
stats: dict[str, int],
error: str | None = None,
cancelled: bool = False,
):
_cancel_cache.pop(run_id, None)
status = "cancelled" if cancelled else ("error" if error else "done")
sql = """ sql = """
UPDATE crawl_run SET UPDATE crawl_run SET
status = %s, finished_at = now(), status = %s, finished_at = now(),
@ -267,7 +322,7 @@ def finish_crawl_run(run_id: int, stats: Dict[str, int], error: Optional[str] =
# Checkpoints # Checkpoints
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def get_checkpoint(key: str) -> Optional[str]: def get_checkpoint(key: str) -> str | None:
with get_conn() as conn: with get_conn() as conn:
with conn.cursor() as cur: with conn.cursor() as cur:
cur.execute("SELECT value FROM crawl_checkpoint WHERE key = %s", (key,)) cur.execute("SELECT value FROM crawl_checkpoint WHERE key = %s", (key,))
@ -290,7 +345,7 @@ def set_checkpoint(key: str, value: str):
# Job triggers (déclenchement depuis le dashboard admin) # Job triggers (déclenchement depuis le dashboard admin)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def claim_job_trigger(job_type: Optional[str] = None) -> Optional[int]: def claim_job_trigger(job_type: str | None = None) -> int | None:
"""Récupère et verrouille un job trigger en attente. Retourne son id ou None.""" """Récupère et verrouille un job trigger en attente. Retourne son id ou None."""
where = "status = 'pending'" where = "status = 'pending'"
params: list = [] params: list = []
@ -317,7 +372,7 @@ def claim_job_trigger(job_type: Optional[str] = None) -> Optional[int]:
return None return None
def finish_job_trigger(trigger_id: int, error: Optional[str] = None): def finish_job_trigger(trigger_id: int, error: str | None = None):
try: try:
with get_conn() as conn: with get_conn() as conn:
with conn.cursor() as cur: with conn.cursor() as cur:
@ -333,7 +388,7 @@ def finish_job_trigger(trigger_id: int, error: Optional[str] = None):
# Helpers # Helpers
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def _parse_year(s: Optional[str]) -> Optional[int]: def _parse_year(s: str | None) -> int | None:
if not s: if not s:
return None return None
m = __import__("re").search(r"\b(1[89]\d\d|20\d\d)\b", str(s)) m = __import__("re").search(r"\b(1[89]\d\d|20\d\d)\b", str(s))

View file

@ -9,7 +9,6 @@ Workflow :
5. fs.destroy_session(session_id) libère la ressource Chrome 5. fs.destroy_session(session_id) libère la ressource Chrome
""" """
import logging import logging
from typing import Optional, Tuple
import requests import requests
@ -47,7 +46,7 @@ class FlareSolverr:
except Exception as e: except Exception as e:
log.warning(f"[fs] destroy session {session_id} failed: {e}") log.warning(f"[fs] destroy session {session_id} failed: {e}")
def get(self, url: str, session_id: Optional[str] = None) -> Tuple[int, str]: def get(self, url: str, session_id: str | None = None) -> tuple[int, str]:
""" """
GET via FlareSolverr. Retourne (http_status_code, response_body). GET via FlareSolverr. Retourne (http_status_code, response_body).
Si session_id fourni, réutilise le Chrome existant (pas de re-warmup). Si session_id fourni, réutilise le Chrome existant (pas de re-warmup).

View file

@ -2,29 +2,35 @@
Jobs de crawl. Chaque fonction est un job indépendant appelé par le scheduler. Jobs de crawl. Chaque fonction est un job indépendant appelé par le scheduler.
""" """
import logging import logging
from datetime import datetime, timezone from datetime import UTC, datetime
from typing import List, Optional
from .config import ( from .config import (
BAND_MIN_DELAY, BAND_MAX_DELAY, BAND_MAX_DELAY,
COOLDOWN_EVERY, COOLDOWN_MIN, COOLDOWN_MAX, BAND_MIN_DELAY,
COOLDOWN_EVERY,
COOLDOWN_MAX,
COOLDOWN_MIN,
MA_BASE, MA_BASE,
) )
from .db import ( from .db import (
upsert_bands, upsert_band_enriched, RunCancelled,
finish_crawl_run,
get_bands_to_enrich, get_bands_to_enrich,
start_crawl_run, finish_crawl_run, get_checkpoint,
update_crawl_run_progress,
get_checkpoint, set_checkpoint,
log_event, log_event,
raise_if_cancelled,
set_checkpoint,
start_crawl_run,
update_crawl_run_progress,
upsert_band_enriched,
upsert_bands,
) )
from .europe_codes import EUROPE_COUNTRY_CODES from .europe_codes import EUROPE_COUNTRY_CODES
from .ma_http import MASession
from .polite import cooldown, sleep_range
from .scraper_band import page_hash, parse_band_page
_EU_SET = set(EUROPE_COUNTRY_CODES) _EU_SET = set(EUROPE_COUNTRY_CODES)
from .europe_codes import EUROPE_COUNTRY_CODES
from .ma_http import MASession
from .polite import sleep_range, cooldown
from .scraper_band import parse_band_page, page_hash
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
@ -35,21 +41,26 @@ CHUNK = 300 # taille des batches d'upsert
# Crawl complet (tous les pays d'un coup) # Crawl complet (tous les pays d'un coup)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def run_full_crawl(session: MASession, countries: List[str] = None): def run_full_crawl(session: MASession, countries: list[str] = None):
countries = countries or EUROPE_COUNTRY_CODES countries = countries or EUROPE_COUNTRY_CODES
run_id = start_crawl_run("full_europe", countries) run_id = start_crawl_run("full_europe", countries)
stats = {"seen": 0, "new": 0, "updated": 0, "enriched": 0} stats = {"seen": 0, "new": 0, "updated": 0, "enriched": 0}
error = None error = None
cancelled = False
log_event("info", f"full crawl started ({len(countries)} countries)", run_id=run_id) log_event("info", f"full crawl started ({len(countries)} countries)", run_id=run_id)
try: try:
for cc in countries: for cc in countries:
raise_if_cancelled(run_id)
log.info(f"[full] country={cc}") log.info(f"[full] country={cc}")
buf = [] buf = []
for band in session.fetch_country_bands(cc): for band in session.fetch_country_bands(cc):
band["data"] = {"url": band.pop("url", None)} band["data"] = {"url": band.pop("url", None)}
buf.append(band) buf.append(band)
if len(buf) >= CHUNK: if len(buf) >= CHUNK:
# Un crawl complet Europe dure des heures : le contrôle doit
# tomber à chaque lot, pas seulement entre deux pays.
raise_if_cancelled(run_id)
r = upsert_bands(buf) r = upsert_bands(buf)
stats["seen"] += len(buf) stats["seen"] += len(buf)
stats["new"] += r["inserted"] stats["new"] += r["inserted"]
@ -64,12 +75,16 @@ def run_full_crawl(session: MASession, countries: List[str] = None):
set_checkpoint("last_full_crawl_at", _now_iso()) set_checkpoint("last_full_crawl_at", _now_iso())
log.info(f"[full] done: {stats}") log.info(f"[full] done: {stats}")
log_event("info", f"full crawl done: {stats}", run_id=run_id) log_event("info", f"full crawl done: {stats}", run_id=run_id)
except RunCancelled:
cancelled = True
log.info(f"[full] annulé sur demande admin: {stats}")
log_event("warning", f"full crawl annulé par un admin: {stats}", run_id=run_id)
except Exception as e: except Exception as e:
error = str(e) error = str(e)
log.error(f"[full] error: {e}", exc_info=True) log.error(f"[full] error: {e}", exc_info=True)
log_event("error", f"full crawl error: {e}", run_id=run_id) log_event("error", f"full crawl error: {e}", run_id=run_id)
finally: finally:
finish_crawl_run(run_id, stats, error) finish_crawl_run(run_id, stats, error, cancelled=cancelled)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
@ -86,6 +101,7 @@ def run_incremental(session: MASession, order_by: str):
run_id = start_crawl_run(f"incremental_{order_by}") run_id = start_crawl_run(f"incremental_{order_by}")
stats = {"seen": 0, "new": 0, "updated": 0, "enriched": 0} stats = {"seen": 0, "new": 0, "updated": 0, "enriched": 0}
error = None error = None
cancelled = False
latest_date = since latest_date = since
log.info(f"[incr/{order_by}] since={since}") log.info(f"[incr/{order_by}] since={since}")
@ -103,6 +119,7 @@ def run_incremental(session: MASession, order_by: str):
band["data"] = {"url": band.pop("url", None)} band["data"] = {"url": band.pop("url", None)}
buf.append(band) buf.append(band)
if len(buf) >= CHUNK: if len(buf) >= CHUNK:
raise_if_cancelled(run_id)
r = upsert_bands(buf) r = upsert_bands(buf)
stats["seen"] += len(buf) stats["seen"] += len(buf)
stats["new"] += r["inserted"] stats["new"] += r["inserted"]
@ -119,19 +136,23 @@ def run_incremental(session: MASession, order_by: str):
set_checkpoint(checkpoint_key, latest_date) set_checkpoint(checkpoint_key, latest_date)
log.info(f"[incr/{order_by}] done: {stats}") log.info(f"[incr/{order_by}] done: {stats}")
log_event("info", f"incremental/{order_by} done: {stats}", run_id=run_id) log_event("info", f"incremental/{order_by} done: {stats}", run_id=run_id)
except RunCancelled:
cancelled = True
log.info(f"[incr/{order_by}] annulé sur demande admin: {stats}")
log_event("warning", f"incremental/{order_by} annulé par un admin: {stats}", run_id=run_id)
except Exception as e: except Exception as e:
error = str(e) error = str(e)
log.error(f"[incr/{order_by}] error: {e}", exc_info=True) log.error(f"[incr/{order_by}] error: {e}", exc_info=True)
log_event("error", f"incremental/{order_by} error: {e}", run_id=run_id) log_event("error", f"incremental/{order_by} error: {e}", run_id=run_id)
finally: finally:
finish_crawl_run(run_id, stats, error) finish_crawl_run(run_id, stats, error, cancelled=cancelled)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Enrichissement des pages individuelles de bands # Enrichissement des pages individuelles de bands
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def run_enrich(session: MASession, limit: int = 100, country: Optional[str] = None): def run_enrich(session: MASession, limit: int = 100, country: str | None = None):
""" """
Visite les pages individuelles des bands non encore enrichies. Visite les pages individuelles des bands non encore enrichies.
Stocke themes, membres, label, dates MA, hash HTML. Stocke themes, membres, label, dates MA, hash HTML.
@ -139,6 +160,7 @@ def run_enrich(session: MASession, limit: int = 100, country: Optional[str] = No
run_id = start_crawl_run("enrich", [country] if country else None) run_id = start_crawl_run("enrich", [country] if country else None)
stats = {"seen": 0, "new": 0, "updated": 0, "enriched": 0} stats = {"seen": 0, "new": 0, "updated": 0, "enriched": 0}
error = None error = None
cancelled = False
try: try:
bands = get_bands_to_enrich(country=country, limit=limit) bands = get_bands_to_enrich(country=country, limit=limit)
@ -146,6 +168,10 @@ def run_enrich(session: MASession, limit: int = 100, country: Optional[str] = No
log_event("info", f"enrich started: {len(bands)} bands queued", run_id=run_id) log_event("info", f"enrich started: {len(bands)} bands queued", run_id=run_id)
for i, band in enumerate(bands): for i, band in enumerate(bands):
# Volontairement AVANT le try/except interne : celui-ci fait
# `except Exception: continue` et avalerait RunCancelled.
raise_if_cancelled(run_id)
url = band.get("url") url = band.get("url")
if not url: if not url:
continue continue
@ -174,12 +200,16 @@ def run_enrich(session: MASession, limit: int = 100, country: Optional[str] = No
log.info(f"[enrich] done: {stats}") log.info(f"[enrich] done: {stats}")
log_event("info", f"enrich done: {stats}", run_id=run_id) log_event("info", f"enrich done: {stats}", run_id=run_id)
except RunCancelled:
cancelled = True
log.info(f"[enrich] annulé sur demande admin: {stats}")
log_event("warning", f"enrich annulé par un admin: {stats}", run_id=run_id)
except Exception as e: except Exception as e:
error = str(e) error = str(e)
log.error(f"[enrich] error: {e}", exc_info=True) log.error(f"[enrich] error: {e}", exc_info=True)
log_event("error", f"enrich error: {e}", run_id=run_id) log_event("error", f"enrich error: {e}", run_id=run_id)
finally: finally:
finish_crawl_run(run_id, stats, error) finish_crawl_run(run_id, stats, error, cancelled=cancelled)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
@ -187,4 +217,4 @@ def run_enrich(session: MASession, limit: int = 100, country: Optional[str] = No
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def _now_iso() -> str: def _now_iso() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%d") return datetime.now(UTC).strftime("%Y-%m-%d")

View file

@ -11,11 +11,11 @@ import logging
import re import re
import time import time
from html import unescape as html_unescape from html import unescape as html_unescape
from typing import Any, Dict, Optional from typing import Any
from urllib.parse import urlencode from urllib.parse import urlencode
from .config import MA_BASE, AJAX_PAGE_SIZE, LIST_MIN_DELAY, LIST_MAX_DELAY from .config import AJAX_PAGE_SIZE, LIST_MAX_DELAY, LIST_MIN_DELAY, MA_BASE
from .flaresolverr import FlareSolverr, FlareSolverrError from .flaresolverr import FlareSolverr
from .polite import sleep_range from .polite import sleep_range
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
@ -27,7 +27,7 @@ _SESSION_TTL = 3600 * 4 # refresh session toutes les 4h
class MASession: class MASession:
def __init__(self, fs: FlareSolverr): def __init__(self, fs: FlareSolverr):
self.fs = fs self.fs = fs
self._session_id: Optional[str] = None self._session_id: str | None = None
self._session_age = 0.0 self._session_age = 0.0
# ------------------------------------------------------------------ # ------------------------------------------------------------------
@ -47,7 +47,7 @@ class MASession:
log.info(f"[ma_http] warmup done (status={status}), session ready") log.info(f"[ma_http] warmup done (status={status}), session ready")
self._session_age = time.time() self._session_age = time.time()
def get_json(self, url: str, params: Dict = None, retries: int = 2) -> Any: def get_json(self, url: str, params: dict = None, retries: int = 2) -> Any:
"""GET JSON depuis un endpoint AJAX MA.""" """GET JSON depuis un endpoint AJAX MA."""
self.ensure_session() self.ensure_session()
full_url = _build_url(url, params) full_url = _build_url(url, params)
@ -68,7 +68,9 @@ class MASession:
self.ensure_session(force_refresh=True) self.ensure_session(force_refresh=True)
sleep_range(LIST_MIN_DELAY * 2, LIST_MAX_DELAY * 2) sleep_range(LIST_MIN_DELAY * 2, LIST_MAX_DELAY * 2)
continue continue
raise RuntimeError(f"JSON parse error après {retries} tentatives sur {url}: {e}") raise RuntimeError(
f"JSON parse error après {retries} tentatives sur {url}: {e}"
) from e
raise RuntimeError(f"Failed after {retries} retries: {url}") raise RuntimeError(f"Failed after {retries} retries: {url}")
def get_html(self, url: str, retries: int = 2) -> str: def get_html(self, url: str, retries: int = 2) -> str:
@ -134,7 +136,7 @@ class MASession:
sleep_range(LIST_MIN_DELAY, LIST_MAX_DELAY) sleep_range(LIST_MIN_DELAY, LIST_MAX_DELAY)
def fetch_archive_bands(self, order_by: str, since_date: Optional[str] = None): def fetch_archive_bands(self, order_by: str, since_date: str | None = None):
""" """
Pagine l'endpoint AJAX des archives (latest additions ou latest modified). Pagine l'endpoint AJAX des archives (latest additions ou latest modified).
order_by : 'created' | 'modified' order_by : 'created' | 'modified'
@ -187,7 +189,7 @@ class MASession:
# Helpers internes # Helpers internes
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def _build_url(base: str, params: Dict = None) -> str: def _build_url(base: str, params: dict = None) -> str:
if not params: if not params:
return base return base
return f"{base}?{urlencode(params)}" return f"{base}?{urlencode(params)}"
@ -215,7 +217,7 @@ def _extract_link(html_str: str):
return href, text return href, text
def _extract_ma_id(url: str) -> Optional[int]: def _extract_ma_id(url: str) -> int | None:
m = re.search(r"/bands/[^/]+/(\d+)", url or "") m = re.search(r"/bands/[^/]+/(\d+)", url or "")
return int(m.group(1)) if m else None return int(m.group(1)) if m else None
@ -224,7 +226,7 @@ def _clean(s) -> str:
return re.sub(r"<[^>]+>", "", str(s or "")).strip() return re.sub(r"<[^>]+>", "", str(s or "")).strip()
def _parse_list_row(row, country_code: str) -> Optional[Dict]: def _parse_list_row(row, country_code: str) -> dict | None:
"""Ligne du listing pays : [name_html, genre, location, status]""" """Ligne du listing pays : [name_html, genre, location, status]"""
if len(row) < 3: if len(row) < 3:
return None return None
@ -243,7 +245,7 @@ def _parse_list_row(row, country_code: str) -> Optional[Dict]:
} }
def _parse_archive_row(row, order_by: str) -> Optional[Dict]: def _parse_archive_row(row, order_by: str) -> dict | None:
""" """
Ligne des archives MA (6 colonnes) : Ligne des archives MA (6 colonnes) :
[date_label, band_link, country_link, genre, time_label, user_link] [date_label, band_link, country_link, genre, time_label, user_link]

View file

@ -16,21 +16,23 @@ Variables d'env :
CRAWLER_FULL_CRAWL_INTERVAL_DAYS (défaut: 60, mettre 0 pour désactiver) CRAWLER_FULL_CRAWL_INTERVAL_DAYS (défaut: 60, mettre 0 pour désactiver)
""" """
import logging import logging
import os
import sys import sys
import time import time
from datetime import datetime, timezone from datetime import UTC, datetime
import schedule import schedule
from .config import ( from .config import (
FLARESOLVERR_URL, FS_TIMEOUT_MS,
SCHED_INCREMENTAL_H, SCHED_ENRICH_H, FULL_CRAWL_INTERVAL_DAYS,
ENRICH_LIMIT, ENRICH_LIMIT,
FLARESOLVERR_URL,
FS_TIMEOUT_MS,
FULL_CRAWL_INTERVAL_DAYS,
SCHED_ENRICH_H,
SCHED_INCREMENTAL_H,
) )
from .flaresolverr import FlareSolverr, FlareSolverrError
from .db import claim_job_trigger, finish_job_trigger, get_checkpoint from .db import claim_job_trigger, finish_job_trigger, get_checkpoint
from .jobs import run_full_crawl, run_incremental, run_enrich from .flaresolverr import FlareSolverr
from .jobs import run_enrich, run_full_crawl, run_incremental
from .ma_http import MASession from .ma_http import MASession
logging.basicConfig( logging.basicConfig(
@ -79,8 +81,8 @@ def main():
last = get_checkpoint("last_full_crawl_at") last = get_checkpoint("last_full_crawl_at")
if last: if last:
try: try:
last_dt = datetime.strptime(last, "%Y-%m-%d").replace(tzinfo=timezone.utc) last_dt = datetime.strptime(last, "%Y-%m-%d").replace(tzinfo=UTC)
elapsed_days = (datetime.now(timezone.utc) - last_dt).days elapsed_days = (datetime.now(UTC) - last_dt).days
if elapsed_days < FULL_CRAWL_INTERVAL_DAYS: if elapsed_days < FULL_CRAWL_INTERVAL_DAYS:
return return
except ValueError: except ValueError:

View file

@ -6,7 +6,7 @@ le schéma band_page historique stocké en DB (lineup structuré, discography_ur
""" """
import hashlib import hashlib
import re import re
from typing import Any, Dict, List, Optional from typing import Any
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
@ -19,16 +19,16 @@ _LINEUP_SECTIONS = {
} }
def parse_band_page(html: str) -> Dict[str, Any]: def parse_band_page(html: str) -> dict[str, Any]:
soup = BeautifulSoup(html, "lxml") soup = BeautifulSoup(html, "lxml")
out: Dict[str, Any] = {} out: dict[str, Any] = {}
h1 = soup.find("h1", class_=re.compile(r"band_name", re.I)) or soup.find("h1") h1 = soup.find("h1", class_=re.compile(r"band_name", re.I)) or soup.find("h1")
out["name"] = _text(h1) out["name"] = _text(h1)
# --- band_stats : dt/dd pairs --- # --- band_stats : dt/dd pairs ---
stats_div = soup.find("div", id="band_stats") or soup.find("div", id="band_info") or soup stats_div = soup.find("div", id="band_stats") or soup.find("div", id="band_info") or soup
kv: Dict[str, str] = {} kv: dict[str, str] = {}
for dl in stats_div.find_all("dl"): for dl in stats_div.find_all("dl"):
for dt in dl.find_all("dt"): for dt in dl.find_all("dt"):
dd = dt.find_next_sibling("dd") dd = dt.find_next_sibling("dd")
@ -50,7 +50,7 @@ def parse_band_page(html: str) -> Dict[str, Any]:
out["label"] = _pick(kv, "Current label", "Last label", "Label") out["label"] = _pick(kv, "Current label", "Last label", "Label")
# --- Lineup (grouped by section, with artist URLs) --- # --- Lineup (grouped by section, with artist URLs) ---
lineup: Dict[str, List[Dict]] = {"current": [], "past": [], "live": [], "session": []} lineup: dict[str, list[dict]] = {"current": [], "past": [], "live": [], "session": []}
for table in soup.find_all("table", class_="lineupTable"): for table in soup.find_all("table", class_="lineupTable"):
section_key = _lineup_section(table) section_key = _lineup_section(table)
bucket = lineup.setdefault(section_key, []) bucket = lineup.setdefault(section_key, [])
@ -104,7 +104,7 @@ def _text(el) -> str:
return (el.get_text(" ", strip=True) if el else "").strip() return (el.get_text(" ", strip=True) if el else "").strip()
def _pick(kv: Dict, *keys: str) -> Optional[str]: def _pick(kv: dict, *keys: str) -> str | None:
for k in keys: for k in keys:
if k in kv: if k in kv:
return kv[k] return kv[k]

View file

@ -0,0 +1,12 @@
import os
import sys
from pathlib import Path
# Le crawler s'exécute en `python -m src.main` depuis /app : les tests doivent
# voir la même racine pour que les imports relatifs (`from .db import ...`)
# fonctionnent à l'identique.
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
# db.py lit DATABASE_URL à l'import via config.py. Aucune connexion n'est
# ouverte tant qu'on n'appelle pas get_conn(), que les tests remplacent.
os.environ.setdefault("DATABASE_URL", "postgresql://test:test@localhost:5432/test")

View file

@ -0,0 +1,172 @@
"""
Annulation coopérative des crawl_run.
C'est la moitié qui fait le travail réel : l'API ne fait que poser un drapeau
(voir apps/api/test/cancellation.test.js), c'est le crawler qui doit l'observer
et s'arrêter. Sans ces tests, le bouton « Annuler » pourrait redevenir
cosmétique sans que rien ne le signale.
Aucune connexion Postgres : `get_conn` est remplacé par un faux curseur.
"""
import contextlib
from unittest.mock import patch
import pytest
from src import db
from src.db import RunCancelled, is_cancel_requested, raise_if_cancelled
class FakeCursor:
"""Curseur minimal qui rejoue une file de résultats et compte les requêtes."""
def __init__(self, results, fail=False):
self._results = list(results)
self.queries = []
self.fail = fail
def execute(self, sql, params=None):
if self.fail:
raise RuntimeError("base injoignable")
self.queries.append((sql, params))
def fetchone(self):
return self._results.pop(0) if self._results else None
def __enter__(self):
return self
def __exit__(self, *a):
return False
def fake_conn(cursor):
@contextlib.contextmanager
def _get_conn():
class Conn:
def cursor(self_inner):
return cursor
yield Conn()
return _get_conn
@pytest.fixture(autouse=True)
def _clear_cache():
"""Le drapeau est mis en cache par run_id : repartir propre à chaque test."""
db._cancel_cache.clear()
yield
db._cancel_cache.clear()
class TestIsCancelRequested:
def test_false_quand_le_drapeau_est_baisse(self):
cur = FakeCursor([(False,)])
with patch.object(db, "get_conn", fake_conn(cur)):
assert is_cancel_requested(1) is False
def test_true_quand_le_drapeau_est_leve(self):
cur = FakeCursor([(True,)])
with patch.object(db, "get_conn", fake_conn(cur)):
assert is_cancel_requested(1) is True
def test_false_quand_le_run_nexiste_pas(self):
cur = FakeCursor([])
with patch.object(db, "get_conn", fake_conn(cur)):
assert is_cancel_requested(999) is False
def test_le_resultat_est_mis_en_cache(self):
"""Le check est appelé à chaque lot : il ne doit pas faire une requête par appel."""
cur = FakeCursor([(False,), (False,), (False,)])
with patch.object(db, "get_conn", fake_conn(cur)):
for _ in range(5):
is_cancel_requested(1)
assert len(cur.queries) == 1
def test_force_contourne_le_cache(self):
cur = FakeCursor([(False,), (True,)])
with patch.object(db, "get_conn", fake_conn(cur)):
assert is_cancel_requested(1) is False
assert is_cancel_requested(1, force=True) is True
assert len(cur.queries) == 2
def test_le_cache_est_par_run(self):
cur = FakeCursor([(True,), (False,)])
with patch.object(db, "get_conn", fake_conn(cur)):
assert is_cancel_requested(1) is True
assert is_cancel_requested(2) is False
# Une base momentanément injoignable ne doit PAS faire passer un crawl
# sain pour un crawl annulé.
def test_une_erreur_db_ne_declenche_pas_dannulation(self):
cur = FakeCursor([], fail=True)
with patch.object(db, "get_conn", fake_conn(cur)):
assert is_cancel_requested(1) is False
def test_une_erreur_db_conserve_la_derniere_valeur_connue(self):
ok = FakeCursor([(True,)])
with patch.object(db, "get_conn", fake_conn(ok)):
assert is_cancel_requested(1) is True
db._cancel_cache[1] = (0.0, True) # cache expiré, valeur connue = True
ko = FakeCursor([], fail=True)
with patch.object(db, "get_conn", fake_conn(ko)):
assert is_cancel_requested(1) is True
def test_interroge_bien_la_colonne_cancel_requested(self):
cur = FakeCursor([(False,)])
with patch.object(db, "get_conn", fake_conn(cur)):
is_cancel_requested(42)
sql, params = cur.queries[0]
assert "cancel_requested" in sql
assert "crawl_run" in sql
assert params == (42,)
class TestRaiseIfCancelled:
def test_ne_leve_pas_quand_le_drapeau_est_baisse(self):
cur = FakeCursor([(False,)])
with patch.object(db, "get_conn", fake_conn(cur)):
raise_if_cancelled(1) # ne doit pas lever
def test_leve_RunCancelled_quand_le_drapeau_est_leve(self):
cur = FakeCursor([(True,)])
with patch.object(db, "get_conn", fake_conn(cur)):
with pytest.raises(RunCancelled):
raise_if_cancelled(1)
# RunCancelled doit rester attrapable par les `except Exception` existants
# tout en étant distinguable — d'où l'ordre des clauses dans jobs.py.
def test_RunCancelled_derive_de_Exception(self):
assert issubclass(RunCancelled, Exception)
class TestFinishCrawlRun:
def _run_finish(self, **kwargs):
cur = FakeCursor([])
with patch.object(db, "get_conn", fake_conn(cur)):
db.finish_crawl_run(1, {"seen": 5}, **kwargs)
return cur.queries[0]
def test_statut_done_par_defaut(self):
_sql, params = self._run_finish()
assert params[0] == "done"
def test_statut_error_avec_une_erreur(self):
_sql, params = self._run_finish(error="boom")
assert params[0] == "error"
# Un run annulé n'est pas un échec : il ne doit pas remonter dans les erreurs.
def test_statut_cancelled_quand_annule(self):
_sql, params = self._run_finish(cancelled=True)
assert params[0] == "cancelled"
def test_cancelled_prime_sur_error(self):
_sql, params = self._run_finish(error="interrompu", cancelled=True)
assert params[0] == "cancelled"
def test_le_cache_dannulation_est_purge(self):
db._cancel_cache[1] = (999.0, True)
self._run_finish(cancelled=True)
assert 1 not in db._cancel_cache

View file

@ -0,0 +1,9 @@
# Le Dockerfile du geocoder fait `COPY . .` : sans ce fichier, tout ce qui
# traîne dans apps/geocoder/ (caches, tests, venv) part dans l'image.
__pycache__/
*.pyc
*.pyo
.pytest_cache/
.ruff_cache/
.venv/
tests/

View file

@ -18,12 +18,12 @@ Tourne en continu (restart: unless-stopped), poll toutes les 15s.
import os import os
import time import time
import psycopg2
import psycopg2
from parser import ( from parser import (
parse_location_text,
country_centroid,
COUNTRY_NAME_TO_ISO2, COUNTRY_NAME_TO_ISO2,
country_centroid,
parse_location_text,
) )
POLL_INTERVAL = int(os.environ.get("GEOCODE_ENQUEUE_POLL", "15")) POLL_INTERVAL = int(os.environ.get("GEOCODE_ENQUEUE_POLL", "15"))

View file

@ -15,15 +15,16 @@ Rate limits Groq free tier :
llama-3.1-8b-instant : 30 req/min, 14 400 req/jour (fallback) llama-3.1-8b-instant : 30 req/min, 14 400 req/jour (fallback)
""" """
import hashlib
import json
import os import os
import re
import sys import sys
import time import time
import json from datetime import UTC, datetime
import hashlib
import re
import requests
import psycopg2 import psycopg2
from datetime import datetime, timezone import requests
# sys.path[0] = src/ quand lancé comme "python src/groq_worker.py" # sys.path[0] = src/ quand lancé comme "python src/groq_worker.py"
from parser import COUNTRY_NAMES from parser import COUNTRY_NAMES
@ -155,7 +156,7 @@ def main():
day_count = {m[0]: 0 for m in MODELS} day_count = {m[0]: 0 for m in MODELS}
min_count = {m[0]: 0 for m in MODELS} min_count = {m[0]: 0 for m in MODELS}
min_start = time.monotonic() min_start = time.monotonic()
day_start = datetime.now(timezone.utc).date() day_start = datetime.now(UTC).date()
with conn.cursor() as cur: with conn.cursor() as cur:
while True: while True:
@ -163,7 +164,7 @@ def main():
if time.monotonic() - min_start >= 60: if time.monotonic() - min_start >= 60:
min_count = {m[0]: 0 for m in MODELS} min_count = {m[0]: 0 for m in MODELS}
min_start = time.monotonic() min_start = time.monotonic()
today = datetime.now(timezone.utc).date() today = datetime.now(UTC).date()
if today != day_start: if today != day_start:
day_count = {m[0]: 0 for m in MODELS} day_count = {m[0]: 0 for m in MODELS}
day_start = today day_start = today

View file

@ -14,13 +14,13 @@ Stratégie :
4. Succès band_locations + sync bands.lat/lon (origine). 4. Succès band_locations + sync bands.lat/lon (origine).
""" """
import os
import time
import random
import json import json
import requests import os
import psycopg2 import random
import time
import psycopg2
import requests
from parser import build_fallback_queries from parser import build_fallback_queries
GEOAPIFY_API_KEY = os.environ.get("GEOAPIFY_API_KEY", "").strip() GEOAPIFY_API_KEY = os.environ.get("GEOAPIFY_API_KEY", "").strip()

View file

@ -0,0 +1,129 @@
"""
Tests du parseur de localisations Metal Archives.
Aucun accès réseau ni base : `parser.py` n'a que des fonctions pures, ce qui en
fait le module le plus rentable à couvrir de tout le pipeline de géocodage
c'est lui qui décide combien de lignes `band_locations` sont créées, et donc
combien d'appels Geoapify/Groq (payants) seront déclenchés.
"""
import pytest
from src.parser import (
_classify,
build_fallback_queries,
country_centroid,
parse_location_text,
)
class TestClassify:
@pytest.mark.parametrize("value", ["N/A", "n/a", "", " "])
def test_valeurs_vides_ignorees(self, value):
assert _classify(value) == "skip"
@pytest.mark.parametrize("value", ["FR", "DE", "NO"])
def test_codes_iso2(self, value):
assert _classify(value) == "country_code"
def test_ville_normale(self):
assert _classify("Oslo") == "parseable"
class TestParseLocationText:
def test_texte_vide_ne_produit_aucune_ligne(self):
assert parse_location_text("") == []
assert parse_location_text("N/A") == []
def test_ville_simple(self):
rows = parse_location_text("Oslo")
assert len(rows) == 1
assert rows[0]["location_raw"] == "Oslo"
assert rows[0]["is_country_only"] is False
assert rows[0]["step_order"] == 0
def test_ville_avec_region(self):
rows = parse_location_text("Bergen, Hordaland")
assert [r["location_raw"] for r in rows] == ["Bergen, Hordaland"]
def test_code_pays_seul_marque_country_only(self):
rows = parse_location_text("FR")
assert len(rows) == 1
assert rows[0]["is_country_only"] is True
assert rows[0]["location_raw"] == "FR"
def test_code_pays_minuscule_non_reconnu(self):
"""Comportement actuel documenté, pas une validation.
`_ISO2_RE` vaut `^[A-Z]{2}$` : un code pays en minuscules n'est PAS
reconnu comme pays et repart en géocodage comme s'il s'agissait d'une
ville, ce qui consomme un appel Geoapify pour un résultat au mieux
douteux. Metal Archives renvoie les codes en majuscules, donc le cas ne
se produit pas en pratique aujourd'hui — mais rien ne le garantit.
Si `_ISO2_RE` devient insensible à la casse, ce test doit être inversé.
"""
rows = parse_location_text("fr")
assert rows[0]["is_country_only"] is False
def test_plusieurs_villes_separees(self):
"""Une même étape peut lister plusieurs villes : chacune devient une ligne."""
rows = parse_location_text("Oslo/Bergen")
assert len(rows) >= 2
assert {r["location_raw"] for r in rows} >= {"Oslo", "Bergen"}
# Même étape → même step_order
assert len({r["step_order"] for r in rows}) == 1
def test_etapes_successives_incrementent_step_order(self):
rows = parse_location_text("Oslo (early), Bergen (later)")
orders = [r["step_order"] for r in rows]
assert orders == sorted(orders)
def test_les_valeurs_ignorables_ne_creent_pas_de_ligne(self):
rows = parse_location_text("N/A")
assert rows == []
def test_resultat_toujours_serialisable(self):
"""Chaque ligne doit exposer exactement les clés attendues par db.py."""
for row in parse_location_text("Trondheim (1991-1993), Oslo"):
assert set(row) == {"step_order", "step_label", "location_raw", "is_country_only"}
assert isinstance(row["step_order"], int)
assert isinstance(row["is_country_only"], bool)
class TestCountryCentroid:
def test_code_iso2_connu(self):
lat, lon = country_centroid("FR")
assert 41 < lat < 52
assert -6 < lon < 10
def test_insensible_a_la_casse_et_aux_espaces(self):
assert country_centroid(" fr ") == country_centroid("FR")
def test_code_inconnu_renvoie_none(self):
assert country_centroid("ZZ") is None
class TestBuildFallbackQueries:
def test_du_plus_specifique_au_plus_vague(self):
queries = build_fallback_queries("Bergen, Hordaland", "NO")
assert queries[0].startswith("Bergen, Hordaland")
assert queries[-1] == "Bergen"
def test_aucun_doublon(self):
queries = build_fallback_queries("Oslo", "NO")
assert len(queries) == len(set(queries))
def test_le_pays_est_injecte_comme_contexte(self):
queries = build_fallback_queries("Bergen", "NO")
assert any("Norway" in q for q in queries)
def test_pays_deja_present_non_duplique(self):
queries = build_fallback_queries("Bergen, Norway", "NO")
assert not any(q.count("Norway") > 1 for q in queries)
def test_pays_inconnu_ne_plante_pas(self):
assert build_fallback_queries("Bergen", "ZZ") == ["Bergen"]
assert build_fallback_queries("Bergen", None) == ["Bergen"]
def test_toujours_au_moins_une_requete(self):
for raw in ["Oslo", "A, B, C, D", "Saint-Étienne"]:
assert len(build_fallback_queries(raw, "FR")) >= 1

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

View file

@ -1,696 +0,0 @@
<!DOCTYPE html>
<html lang="fr">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>✨ Le Quiz des Équipes</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link href="https://fonts.googleapis.com/css2?family=Fraunces:ital,wght@0,700;0,900;1,700&family=Nunito:wght@400;600;700;800&display=swap" rel="stylesheet">
<style>
:root {
--bg: #0f0e17;
--surface: #1a1929;
--border: rgba(255,255,255,0.08);
--text: #fffffe;
--muted: rgba(255,255,255,0.45);
--saiga: #f7b731;
--axolotl: #26d0ce;
--ratel: #ff6b6b;
--poule: #ffd166;
--echidne: #c77dff;
--accent: #ff6b6b;
}
* { box-sizing: border-box; margin: 0; padding: 0; }
body {
background: var(--bg);
color: var(--text);
font-family: 'Nunito', sans-serif;
min-height: 100vh;
display: flex;
align-items: center;
justify-content: center;
overflow-x: hidden;
}
body::before {
content: '';
position: fixed;
inset: 0;
background:
radial-gradient(ellipse 60% 50% at 20% 10%, rgba(199,125,255,0.12) 0%, transparent 60%),
radial-gradient(ellipse 50% 40% at 80% 90%, rgba(255,107,107,0.1) 0%, transparent 60%);
pointer-events: none;
z-index: 0;
}
.app {
position: relative;
z-index: 1;
width: 100%;
max-width: 560px;
padding: 1.5rem 1.25rem 2rem;
}
/* ─── SCREEN TRANSITIONS ─── */
.screen {
display: none;
animation: fadeUp .45s cubic-bezier(.22,1,.36,1) both;
}
.screen.active { display: block; }
@keyframes fadeUp {
from { opacity: 0; transform: translateY(28px); }
to { opacity: 1; transform: translateY(0); }
}
/* ─── HEADER ─── */
.quiz-header {
text-align: center;
margin-bottom: 2rem;
}
.quiz-header .eyebrow {
font-size: .75rem;
letter-spacing: .18em;
text-transform: uppercase;
color: var(--muted);
margin-bottom: .4rem;
}
.quiz-header h1 {
font-family: 'Fraunces', serif;
font-size: clamp(2rem, 8vw, 2.8rem);
font-weight: 900;
line-height: 1.05;
background: linear-gradient(135deg, #fff 30%, var(--echidne));
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
background-clip: text;
}
.quiz-header p {
margin-top: .6rem;
color: var(--muted);
font-size: .9rem;
line-height: 1.5;
}
/* ─── NAME SELECTION ─── */
.search-wrap {
position: relative;
margin-bottom: 1.2rem;
}
.search-wrap input {
width: 100%;
background: var(--surface);
border: 1.5px solid var(--border);
border-radius: 14px;
padding: .85rem 1.1rem .85rem 2.8rem;
color: var(--text);
font-family: 'Nunito', sans-serif;
font-size: 1rem;
font-weight: 600;
outline: none;
transition: border-color .2s;
}
.search-wrap input:focus { border-color: rgba(199,125,255,.5); }
.search-wrap input::placeholder { color: var(--muted); font-weight: 400; }
.search-icon {
position: absolute;
left: .95rem;
top: 50%;
transform: translateY(-50%);
color: var(--muted);
font-size: 1rem;
pointer-events: none;
}
.names-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(130px, 1fr));
gap: .55rem;
max-height: 58vh;
overflow-y: auto;
padding-right: .2rem;
}
.names-grid::-webkit-scrollbar { width: 4px; }
.names-grid::-webkit-scrollbar-track { background: transparent; }
.names-grid::-webkit-scrollbar-thumb { background: rgba(255,255,255,.15); border-radius: 4px; }
.name-btn {
background: var(--surface);
border: 1.5px solid var(--border);
border-radius: 12px;
padding: .7rem .9rem;
color: var(--text);
font-family: 'Nunito', sans-serif;
font-size: .88rem;
font-weight: 700;
cursor: pointer;
text-align: left;
transition: all .18s;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.name-btn:hover {
border-color: rgba(199,125,255,.5);
background: rgba(199,125,255,.08);
transform: translateY(-1px);
}
.name-btn:active { transform: scale(.96); }
.no-results {
text-align: center;
color: var(--muted);
padding: 2rem 0;
font-size: .9rem;
}
/* ─── QUIZ ─── */
.progress-track {
background: rgba(255,255,255,.08);
border-radius: 99px;
height: 4px;
margin-bottom: 2rem;
overflow: hidden;
}
.progress-fill {
height: 100%;
background: linear-gradient(90deg, var(--echidne), var(--ratel));
border-radius: 99px;
transition: width .4s cubic-bezier(.22,1,.36,1);
}
.question-meta {
font-size: .75rem;
letter-spacing: .12em;
text-transform: uppercase;
color: var(--muted);
margin-bottom: .7rem;
}
.question-text {
font-family: 'Fraunces', serif;
font-size: clamp(1.25rem, 5vw, 1.6rem);
font-weight: 700;
line-height: 1.25;
margin-bottom: 1.6rem;
}
.choices {
display: flex;
flex-direction: column;
gap: .65rem;
}
.choice-btn {
background: var(--surface);
border: 1.5px solid var(--border);
border-radius: 14px;
padding: .95rem 1.15rem;
color: var(--text);
font-family: 'Nunito', sans-serif;
font-size: .9rem;
font-weight: 600;
cursor: pointer;
text-align: left;
display: flex;
align-items: flex-start;
gap: .75rem;
transition: all .18s;
line-height: 1.4;
}
.choice-btn .letter {
font-size: .75rem;
font-weight: 800;
letter-spacing: .06em;
color: var(--muted);
background: rgba(255,255,255,.07);
border-radius: 6px;
padding: .2rem .45rem;
flex-shrink: 0;
margin-top: .05rem;
}
.choice-btn:hover {
border-color: rgba(199,125,255,.5);
background: rgba(199,125,255,.07);
transform: translateX(4px);
}
.choice-btn:active { transform: scale(.97); }
.choice-btn.selected {
border-color: var(--echidne);
background: rgba(199,125,255,.13);
}
.choice-btn.selected .letter {
background: var(--echidne);
color: var(--bg);
}
.quiz-nav {
display: flex;
justify-content: flex-end;
margin-top: 1.4rem;
}
.btn-next {
background: var(--text);
color: var(--bg);
border: none;
border-radius: 12px;
padding: .8rem 1.8rem;
font-family: 'Nunito', sans-serif;
font-size: .9rem;
font-weight: 800;
cursor: pointer;
transition: all .18s;
opacity: 0;
pointer-events: none;
}
.btn-next.visible { opacity: 1; pointer-events: all; }
.btn-next:hover { transform: scale(1.04); }
.btn-next:active { transform: scale(.97); }
/* ─── RESULT ─── */
.result-screen {
text-align: center;
padding-top: .5rem;
}
.result-avatar-wrap {
width: 200px;
height: 200px;
border-radius: 50%;
margin: 0 auto 1.2rem;
padding: 3px;
animation: popIn .5s cubic-bezier(.34,1.56,.64,1) .1s both;
/* border color set inline via JS */
}
.result-avatar {
width: 100%;
height: 100%;
border-radius: 50%;
object-fit: cover;
display: block;
background: var(--surface);
}
@keyframes popIn {
from { opacity: 0; transform: scale(.4) rotate(-10deg); }
to { opacity: 1; transform: scale(1) rotate(0deg); }
}
.result-greeting {
font-size: .85rem;
letter-spacing: .1em;
text-transform: uppercase;
color: var(--muted);
margin-bottom: .3rem;
animation: fadeUp .4s .2s both;
}
.result-name {
font-family: 'Fraunces', serif;
font-size: 1.4rem;
font-weight: 700;
margin-bottom: 1rem;
animation: fadeUp .4s .25s both;
}
.result-badge {
display: inline-flex;
align-items: center;
gap: .5rem;
padding: .5rem 1.4rem;
border-radius: 99px;
font-family: 'Fraunces', serif;
font-size: 1.8rem;
font-weight: 900;
letter-spacing: .03em;
margin-bottom: 1.4rem;
animation: fadeUp .4s .3s both;
}
.result-desc {
font-size: 1rem;
line-height: 1.65;
color: rgba(255,255,255,.8);
max-width: 360px;
margin: 0 auto 1.6rem;
animation: fadeUp .4s .35s both;
font-style: italic;
font-family: 'Fraunces', serif;
}
.btn-restart {
background: transparent;
border: 1.5px solid var(--border);
border-radius: 12px;
padding: .75rem 1.6rem;
color: var(--muted);
font-family: 'Nunito', sans-serif;
font-size: .85rem;
font-weight: 700;
cursor: pointer;
transition: all .18s;
animation: fadeUp .4s .5s both;
}
.btn-restart:hover { color: var(--text); border-color: rgba(255,255,255,.3); }
/* ─── CONFETTI ─── */
.confetti-container {
position: fixed;
inset: 0;
pointer-events: none;
overflow: hidden;
z-index: 10;
}
.confetto {
position: absolute;
width: 9px;
height: 9px;
border-radius: 2px;
top: -20px;
animation: confettiFall linear forwards;
}
@keyframes confettiFall {
0% { transform: translateY(0) rotate(0deg); opacity: 1; }
80% { opacity: 1; }
100% { transform: translateY(105vh) rotate(720deg); opacity: 0; }
}
</style>
</head>
<body>
<div class="confetti-container" id="confetti"></div>
<div class="app">
<!-- ═══ SCREEN 1: NAME SELECTION ═══ -->
<div class="screen active" id="screen-name">
<div class="quiz-header">
<div class="eyebrow">🎉 Weekend Anniversaire</div>
<h1>Qui es-tu<br>ce weekend ?</h1>
<p>Sélectionne ton prénom pour découvrir<br>dans quelle équipe tu es.</p>
</div>
<div class="search-wrap">
<span class="search-icon">🔍</span>
<input type="text" id="name-search" placeholder="Recherche ton prénom…" autocomplete="off" />
</div>
<div class="names-grid" id="names-grid"></div>
</div>
<!-- ═══ SCREEN 2: QUIZ ═══ -->
<div class="screen" id="screen-quiz">
<div class="progress-track">
<div class="progress-fill" id="progress-fill" style="width:0%"></div>
</div>
<div class="question-meta" id="q-meta"></div>
<div class="question-text" id="q-text"></div>
<div class="choices" id="q-choices"></div>
<div class="quiz-nav">
<button class="btn-next" id="btn-next">Suivant →</button>
</div>
</div>
<!-- ═══ SCREEN 3: RESULT ═══ -->
<div class="screen" id="screen-result">
<div class="result-screen">
<div class="result-avatar-wrap" id="r-avatar-wrap">
<img class="result-avatar" id="r-avatar" src="" alt="" />
</div>
<div class="result-greeting">Bienvenue dans l'équipe</div>
<div class="result-badge" id="r-badge"></div>
<div class="result-name" id="r-name"></div>
<div class="result-desc" id="r-desc"></div>
<button class="btn-restart" id="btn-restart">← Recommencer</button>
</div>
</div>
</div>
<script>
// ══════════════════════════════════════════
// DATA
// ══════════════════════════════════════════
const TEAMS = {
SAÏGA: {
img: 'saiga.jpeg',
color: '#f7b731',
desc: "Cerveau en 4D. Trop d'analyse, pas assez de repos.",
members: ['Kawthar','Alexandra','Yoann','Hugo','Tung','Antoine GAB','Aldéric']
},
AXOLOTL: {
img: 'axolotl.jpeg',
color: '#26d0ce',
desc: 'Tu gères ton chaos avec une grâce étrange. Adapté·e, mais à quel prix.',
members: ['Charlène','Lina','Cécile','Philipp','Léon','Alex Clarisse']
},
RATEL: {
img: 'ratel.jpeg',
color: '#ff6b6b',
desc: 'Agent du chaos. Tu fonces, tu vois après. Et souvent… ça passe.',
members: ['Antoine LAMB','Baptiste JAOU','Baptiste JEAN','Astrid','Thimo','Tiphaine','Floriane','Clarisse BA','Marine','Léonie']
},
'POULE DE SOIE': {
img: 'poule-de-soie.jpeg',
color: '#ffd166',
desc: 'Doux·ce, perdu·e, essentiel·le. Tu tiens le groupe sans le savoir.',
members: ['Laurine','Alice','Indra','Zhong','Laurie','Hawa','Margaux','Alexandre P','Thijs']
},
ECHIDNÉ: {
img: 'echidne.jpeg',
color: '#c77dff',
desc: 'Tu veux juste survivre tranquillement. Soft extérieur, panique intérieure.',
members: ['Emma','Clarisse Authier','Christophe','Alexander','Mélanie','Elsa','Samuel','Tes']
}
};
// Build name → team map
const NAME_TO_TEAM = {};
Object.entries(TEAMS).forEach(([teamName, data]) => {
data.members.forEach(m => NAME_TO_TEAM[m] = teamName);
});
const ALL_NAMES = Object.values(TEAMS).flatMap(t => t.members).sort((a,b) => a.localeCompare(b,'fr'));
const QUESTIONS = [
{
text: 'Ton pire cauchemar',
choices: [
'Être coincé·e dans un call Teams sans fin',
'Outlook qui plante juste avant un envoi crucial',
'Une notif Slack à 23h "petite question rapide"',
'Devoir "networker" avec des gens motivés',
"Que quelqu'un dise \"on fait un tour de table\""
]
},
{
text: 'Ta maison à Poudlard',
choices: [
'Serpentard (ambition + fatigue)',
'Poufsouffle (gentil·le mais épuisé·e)',
'Gryffondor (audace discutable)',
'Serdaigle (cerveau en surchauffe)',
'Refuse de choisir, trop de pression'
]
},
{
text: 'Ton plaisir coupable',
choices: [
'Annuler des plans',
'Regarder des apparts hors budget',
'Acheter du matos de sport inutile',
'Dire "je me couche tôt" puis scroller 2h',
'Regarder des gens mieux organisés que toi'
]
},
{
text: 'Ton état un dimanche soir',
choices: [
'Mélancolie élégante',
'Angoisse administrative',
'Déni total',
'Organisation excessive',
'Snack + série + oubli volontaire'
]
},
{
text: 'Quel fromage es-tu ?',
choices: [
'Comté (fiable, apprécié·e, un peu contrôlant·e)',
'Reblochon (réconfortant·e mais intense)',
'Camembert (classique mais imprévisible)',
'Roquefort (clivant·e, puissant·e, assumé·e)',
'Chèvre frais (léger·ère, sensible, un peu fragile)'
]
}
];
// ══════════════════════════════════════════
// STATE
// ══════════════════════════════════════════
let currentName = '';
let currentQ = 0;
let selectedChoice = null;
// ══════════════════════════════════════════
// SCREENS
// ══════════════════════════════════════════
function showScreen(id) {
document.querySelectorAll('.screen').forEach(s => s.classList.remove('active'));
const el = document.getElementById(id);
el.classList.add('active');
}
// ══════════════════════════════════════════
// NAME SELECTION
// ══════════════════════════════════════════
function renderNames(filter = '') {
const grid = document.getElementById('names-grid');
const filtered = filter
? ALL_NAMES.filter(n => n.toLowerCase().includes(filter.toLowerCase()))
: ALL_NAMES;
if (!filtered.length) {
grid.innerHTML = '<div class="no-results">Aucun prénom trouvé 😅</div>';
return;
}
grid.innerHTML = filtered.map(name =>
`<button class="name-btn" data-name="${name}">${name}</button>`
).join('');
grid.querySelectorAll('.name-btn').forEach(btn => {
btn.addEventListener('click', () => {
currentName = btn.dataset.name;
startQuiz();
});
});
}
document.getElementById('name-search').addEventListener('input', e => {
renderNames(e.target.value);
});
renderNames();
// ══════════════════════════════════════════
// QUIZ
// ══════════════════════════════════════════
function startQuiz() {
currentQ = 0;
selectedChoice = null;
showScreen('screen-quiz');
renderQuestion();
}
function renderQuestion() {
const q = QUESTIONS[currentQ];
const total = QUESTIONS.length;
document.getElementById('progress-fill').style.width = `${(currentQ / total) * 100}%`;
document.getElementById('q-meta').textContent = `Question ${currentQ + 1} / ${total}`;
document.getElementById('q-text').textContent = q.text;
const letters = ['A','B','C','D','E'];
document.getElementById('q-choices').innerHTML = q.choices.map((c, i) =>
`<button class="choice-btn" data-idx="${i}">
<span class="letter">${letters[i]}</span>
<span>${c}</span>
</button>`
).join('');
selectedChoice = null;
const btnNext = document.getElementById('btn-next');
btnNext.classList.remove('visible');
btnNext.textContent = currentQ < QUESTIONS.length - 1 ? 'Suivant ' : 'Révéler mon équipe ';
// Animate in
const choicesEl = document.getElementById('q-choices');
choicesEl.style.opacity = '0';
choicesEl.style.transform = 'translateY(14px)';
setTimeout(() => {
choicesEl.style.transition = 'opacity .3s, transform .3s';
choicesEl.style.opacity = '1';
choicesEl.style.transform = 'translateY(0)';
}, 20);
document.querySelectorAll('.choice-btn').forEach(btn => {
btn.addEventListener('click', () => {
document.querySelectorAll('.choice-btn').forEach(b => b.classList.remove('selected'));
btn.classList.add('selected');
selectedChoice = parseInt(btn.dataset.idx);
btnNext.classList.add('visible');
});
});
}
document.getElementById('btn-next').addEventListener('click', () => {
if (selectedChoice === null) return;
currentQ++;
if (currentQ < QUESTIONS.length) {
renderQuestion();
} else {
showResult();
}
});
// ══════════════════════════════════════════
// RESULT
// ══════════════════════════════════════════
function showResult() {
const teamName = NAME_TO_TEAM[currentName] || 'AXOLOTL';
const team = TEAMS[teamName];
document.getElementById('progress-fill').style.width = '100%';
const avatar = document.getElementById('r-avatar');
avatar.src = team.img;
avatar.alt = teamName;
const wrap = document.getElementById('r-avatar-wrap');
wrap.style.background = `conic-gradient(${team.color} 0%, ${team.color} 100%)`;
wrap.style.boxShadow = `0 0 28px ${team.color}44`;
document.getElementById('r-name').textContent = `Bienvenue, ${currentName} !`;
const badge = document.getElementById('r-badge');
badge.textContent = teamName;
badge.style.background = team.color + '22';
badge.style.border = `2px solid ${team.color}66`;
badge.style.color = team.color;
document.getElementById('r-desc').textContent = `"${team.desc}"`;
showScreen('screen-result');
launchConfetti(team.color);
}
// ══════════════════════════════════════════
// CONFETTI
// ══════════════════════════════════════════
function launchConfetti(mainColor) {
const container = document.getElementById('confetti');
container.innerHTML = '';
const colors = [mainColor, '#fff', '#ffd166', '#c77dff', '#26d0ce', '#ff6b6b'];
const count = 80;
for (let i = 0; i < count; i++) {
const el = document.createElement('div');
el.className = 'confetto';
el.style.left = `${Math.random() * 100}vw`;
el.style.width = el.style.height = `${6 + Math.random() * 8}px`;
el.style.borderRadius = Math.random() > .5 ? '50%' : '2px';
el.style.background = colors[Math.floor(Math.random() * colors.length)];
el.style.opacity = (.6 + Math.random() * .4).toString();
const dur = 1.4 + Math.random() * 1.6;
const delay = Math.random() * .8;
el.style.animation = `confettiFall ${dur}s ${delay}s linear forwards`;
container.appendChild(el);
}
setTimeout(() => { container.innerHTML = ''; }, 4000);
}
// ══════════════════════════════════════════
// RESTART
// ══════════════════════════════════════════
document.getElementById('btn-restart').addEventListener('click', () => {
currentName = '';
document.getElementById('name-search').value = '';
renderNames();
showScreen('screen-name');
});
</script>
</body>
</html>

Binary file not shown.

Before

Width:  |  Height:  |  Size: 554 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 119 KiB

File diff suppressed because it is too large Load diff

View file

@ -48,18 +48,18 @@
<span class="lang-code">EN</span> <span class="lang-code">EN</span>
<span class="lang-arrow"></span> <span class="lang-arrow"></span>
</button> </button>
<div class="lang-options hidden" id="langOptions" role="listbox"> <div class="lang-options hidden" id="langOptions" role="listbox" aria-label="Choisir la langue">
<button class="lang-opt" type="button" data-lang="en"><span class="fi fi-gb"></span>English</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="en"><span class="fi fi-gb"></span>English</button>
<button class="lang-opt" type="button" data-lang="fr"><span class="fi fi-fr"></span>Français</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="fr"><span class="fi fi-fr"></span>Français</button>
<button class="lang-opt" type="button" data-lang="de"><span class="fi fi-de"></span>Deutsch</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="de"><span class="fi fi-de"></span>Deutsch</button>
<button class="lang-opt" type="button" data-lang="es"><span class="fi fi-es"></span>Español</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="es"><span class="fi fi-es"></span>Español</button>
<button class="lang-opt" type="button" data-lang="it"><span class="fi fi-it"></span>Italiano</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="it"><span class="fi fi-it"></span>Italiano</button>
<button class="lang-opt" type="button" data-lang="pl"><span class="fi fi-pl"></span>Polski</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="pl"><span class="fi fi-pl"></span>Polski</button>
<button class="lang-opt" type="button" data-lang="nl"><span class="fi fi-nl"></span>Nederlands</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="nl"><span class="fi fi-nl"></span>Nederlands</button>
<button class="lang-opt" type="button" data-lang="ro"><span class="fi fi-ro"></span>Română</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="ro"><span class="fi fi-ro"></span>Română</button>
<button class="lang-opt" type="button" data-lang="pt"><span class="fi fi-pt"></span>Português</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="pt"><span class="fi fi-pt"></span>Português</button>
<button class="lang-opt" type="button" data-lang="cs"><span class="fi fi-cz"></span>Čeština</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="cs"><span class="fi fi-cz"></span>Čeština</button>
<button class="lang-opt" type="button" data-lang="sv"><span class="fi fi-se"></span>Svenska</button> <button class="lang-opt" type="button" role="option" aria-selected="false" data-lang="sv"><span class="fi fi-se"></span>Svenska</button>
</div> </div>
</div> </div>
<a href="https://discord.gg/P2DE57Ab" target="_blank" rel="noopener noreferrer" class="social-link" title="Rejoindre notre Discord"> <a href="https://discord.gg/P2DE57Ab" target="_blank" rel="noopener noreferrer" class="social-link" title="Rejoindre notre Discord">
@ -257,6 +257,8 @@
</div> </div>
<script src="./locales.js"></script> <script src="./locales.js"></script>
<!-- pure.js définit la globale BMPure : doit être chargé avant app.js -->
<script src="./pure.js"></script>
<script src="./app.js"></script> <script src="./app.js"></script>
</body> </body>
</html> </html>

149
apps/web/site/pure.js Normal file
View file

@ -0,0 +1,149 @@
/**
* Logique métier pure du site public : normalisation, filtrage, tri.
*
* Extraite d'app.js pour être testable. Toutes les fonctions ici sont sans
* effet de bord et ne lisent AUCUN état global : ce qu'elles filtrent leur est
* passé en argument. app.js conserve l'état (les Map de cases cochées) et
* délègue la décision à ce module.
*
* Chargé par <script> avant app.js (donc via la globale `BMPure`), et importé
* en CommonJS par les tests d'où le pied de fichier. Pas d'étape de build.
*/
// `var` volontaire (et non const) : dans un script classique, seul `var` crée
// une propriété sur l'objet global. app.js s'en sortirait avec `const`, mais
// pas le harnais de test, qui évalue ce fichier dans un contexte `vm` et lit
// `sandbox.BMPure` — exactement comme le navigateur lit `window.BMPure`.
// eslint-disable-next-line no-var
var BMPure = (function () {
"use strict";
function escapeHtml(s) {
return String(s ?? "").replace(
/[&<>"']/g,
(c) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#039;" }[c])
);
}
function norm(s) {
if (s == null || s === "") return "";
return String(s).trim();
}
/** Extrait une année d'un nombre ou d'un texte libre ("formé en 1991"). */
function parseYear(val) {
if (val == null) return null;
const n = Number(val);
if (Number.isFinite(n)) return n;
const m = String(val).match(/(19|20)\d{2}/);
return m ? Number(m[0]) : null;
}
function splitThemes(val) {
if (!val) return [];
if (Array.isArray(val)) return val.map(norm).filter(Boolean);
return String(val).split(/[,;/]/).map(norm).filter(Boolean);
}
function matchesQuery(b, q) {
if (!q) return true;
const themes = Array.isArray(b.themes) ? b.themes.join(" ") : "";
const hay = `${b.name} ${b.genre} ${b.status} ${b.country} ${b.location_text} ${themes}`.toLowerCase();
return hay.includes(q);
}
/**
* Logique tri-état partagée par les filtres statut / pays / genre.
*
* Une valeur ABSENTE de la Map passe (la facette n'existe pas dans le jeu de
* données courant : on ne veut pas masquer des groupes à cause d'une facette
* inconnue). Une valeur présente ne passe que si elle est à true.
* Les valeurs vides retombent sur une clé de repli ("Unknown", "??").
*/
function matchesFacet(rawValue, enabledMap, fallbackKey) {
const v = norm(rawValue);
const key = v || fallbackKey;
if (!enabledMap.has(key)) return true;
return enabledMap.get(key) === true;
}
function matchesMacroGenre(b, macroGenres, macroGenreEnabled, macroActiveCount) {
if (macroActiveCount === 0) return true;
const g = norm(b.genre).toLowerCase();
if (!g) return false;
return macroGenres.some(
(m) => macroGenreEnabled.get(m.key) === true && m.terms.some((t) => g.includes(t))
);
}
function matchesTheme(b, themeEnabled) {
if (!themeEnabled.size) return true;
const allOn = Array.from(themeEnabled.values()).every((v) => v === true);
if (allOn) return true;
const tokens = Array.isArray(b.themes) ? b.themes : [];
if (!tokens.length) return false;
return tokens.some((t) => themeEnabled.get(t) === true);
}
/**
* Un groupe sans année reste visible tant que l'utilisateur n'a pas resserré
* la plage ; dès qu'il la resserre, l'absence d'année devient un motif
* d'exclusion (sinon la sélection « 1990-1995 » ramènerait des groupes sans
* date, ce qui est incompréhensible).
*/
function matchesYear(b, yearFilter, yearRange) {
if (yearFilter.min == null || yearFilter.max == null) return true;
if (yearRange.min == null || yearRange.max == null) return true;
const isFullRange = yearFilter.min === yearRange.min && yearFilter.max === yearRange.max;
if (isFullRange) return true;
const y = b.formed_year;
if (!Number.isFinite(y)) return false;
return y >= yearFilter.min && y <= yearFilter.max;
}
/** Clé d'agrégation des groupes partageant exactement les mêmes coordonnées. */
function keyFromLatLon(lat, lon) {
return `${Number(lat).toFixed(6)},${Number(lon).toFixed(6)}`;
}
const byName = (x, y) =>
(x.name || "").localeCompare(y.name || "", "fr", { sensitivity: "base" });
/** Tri stable, sans effet de bord : renvoie une nouvelle liste. */
function sortBands(arr, mode) {
const a = [...arr];
const byField = (f) => (x, y) =>
(x[f] || "").localeCompare(y[f] || "", "fr", { sensitivity: "base" }) || byName(x, y);
if (mode === "status") a.sort(byField("status"));
else if (mode === "genre") a.sort(byField("genre"));
else if (mode === "country") a.sort(byField("country"));
else if (mode === "year") {
// Les groupes sans année partent en fin de liste (-1).
a.sort(
(x, y) =>
(Number.isFinite(y.formed_year) ? y.formed_year : -1) -
(Number.isFinite(x.formed_year) ? x.formed_year : -1)
);
} else a.sort(byName);
return a;
}
return {
escapeHtml,
norm,
parseYear,
splitThemes,
matchesQuery,
matchesFacet,
matchesMacroGenre,
matchesTheme,
matchesYear,
keyFromLatLon,
sortBands,
};
})();
if (typeof module !== "undefined" && module.exports) module.exports = BMPure;

View file

@ -0,0 +1,96 @@
// @vitest-environment jsdom
import { describe, it, expect, beforeAll } from "vitest";
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import axe from "axe-core";
/**
* Tests d'accessibilité statiques.
*
* On charge le HTML livré tel quel dans jsdom et on passe axe-core dessus.
* Pas de navigateur, pas de conteneur : ~1s pour les deux pages.
*
* Limite assumée : les règles qui nécessitent un vrai moteur de rendu
* (contraste de couleurs, cibles tactiles) ne peuvent pas s'évaluer sous jsdom
* et sont désactivées. Elles ne sont donc PAS couvertes ici voir le README
* de la CI pour l'audit Lighthouse manuel qui les complète.
*/
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = path.resolve(__dirname, "../../..");
const PAGES = {
"site public": path.join(REPO_ROOT, "apps/web/site/index.html"),
"dashboard admin": path.join(REPO_ROOT, "apps/admin/site/index.html"),
};
// Règles inapplicables sous jsdom (pas de layout ni de couleurs calculées).
const DISABLED_RULES = {
"color-contrast": { enabled: false },
"target-size": { enabled: false },
};
async function analyse(htmlPath) {
const html = fs.readFileSync(htmlPath, "utf8");
// On remplace le document entier pour conserver <html lang> et <head>.
document.documentElement.innerHTML = html
.replace(/<!doctype html>/i, "")
.replace(/<\/?html[^>]*>/gi, "");
const langMatch = html.match(/<html[^>]*\blang="([^"]+)"/i);
if (langMatch) document.documentElement.setAttribute("lang", langMatch[1]);
return axe.run(document, {
rules: DISABLED_RULES,
resultTypes: ["violations"],
});
}
describe.each(Object.entries(PAGES))("a11y — %s", (label, htmlPath) => {
let results;
beforeAll(async () => {
results = await analyse(htmlPath);
}, 30000);
it("n'a aucune violation critique ou sérieuse", () => {
const blocking = results.violations.filter((v) => ["critical", "serious"].includes(v.impact));
const summary = blocking.map(
(v) => `[${v.impact}] ${v.id}: ${v.help}\n ${v.nodes.map((n) => n.target.join(" ")).join("\n ")}`
);
expect(summary, `${label} — violations bloquantes :\n${summary.join("\n")}`).toEqual([]);
});
it("déclare une langue de document", () => {
expect(document.documentElement.getAttribute("lang")).toBeTruthy();
});
it("n'utilise pas de tabindex positif (casse l'ordre de tabulation)", () => {
const positive = [...document.querySelectorAll("[tabindex]")].filter(
(el) => Number(el.getAttribute("tabindex")) > 0
);
expect(positive.map((el) => el.outerHTML.slice(0, 80))).toEqual([]);
});
it("donne un nom accessible à chaque contrôle de formulaire", () => {
const unnamed = [...document.querySelectorAll("input, select, textarea")]
.filter((el) => el.type !== "hidden")
.filter((el) => {
const id = el.getAttribute("id");
return !(
el.getAttribute("aria-label") ||
el.getAttribute("aria-labelledby") ||
el.getAttribute("title") ||
el.closest("label") ||
(id && document.querySelector(`label[for="${CSS.escape(id)}"]`))
);
});
expect(unnamed.map((el) => el.outerHTML.slice(0, 100))).toEqual([]);
});
it("chaque image porte un attribut alt", () => {
const noAlt = [...document.querySelectorAll("img")].filter((el) => !el.hasAttribute("alt"));
expect(noAlt.map((el) => el.getAttribute("src"))).toEqual([]);
});
});

339
apps/web/test/pure.test.js Normal file
View file

@ -0,0 +1,339 @@
import { describe, it, expect } from "vitest";
import fs from "node:fs";
import path from "node:path";
import vm from "node:vm";
import { fileURLToPath } from "node:url";
// pure.js est un script classique chargé par <script>, pas un module ES.
// On l'évalue exactement comme le fait le navigateur — ça teste au passage que
// le fichier livré expose bien la globale BMPure attendue par app.js.
const SITE = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../site");
const source = fs.readFileSync(path.join(SITE, "pure.js"), "utf8");
const sandbox = { module: undefined };
vm.createContext(sandbox);
vm.runInContext(source, sandbox, { filename: "pure.js" });
const P = sandbox.BMPure;
describe("chargement du script", () => {
it("expose la globale BMPure attendue par app.js", () => {
expect(P).toBeDefined();
});
it("expose toutes les fonctions auxquelles app.js délègue", () => {
const appJs = fs.readFileSync(path.join(SITE, "app.js"), "utf8");
const used = [...appJs.matchAll(/BMPure\.(\w+)/g)].map((m) => m[1]);
expect(used.length).toBeGreaterThan(0);
for (const fn of new Set(used)) {
expect(typeof P[fn], `BMPure.${fn} utilisé dans app.js`).toBe("function");
}
});
});
const band = (over = {}) => ({
ma_id: 1, name: "Mayhem", genre: "Black Metal", status: "Active",
country: "NO", location_text: "Oslo", themes: [], formed_year: 1984, ...over,
});
describe("escapeHtml", () => {
it("neutralise les caractères d'injection HTML", () => {
expect(P.escapeHtml('<script>alert("x")</script>'))
.toBe("&lt;script&gt;alert(&quot;x&quot;)&lt;/script&gt;");
});
it("échappe l'apostrophe (attribut délimité par des quotes simples)", () => {
expect(P.escapeHtml("l'enfer")).toBe("l&#039;enfer");
});
it("échappe l'esperluette en premier (pas de double échappement inversé)", () => {
expect(P.escapeHtml("&lt;")).toBe("&amp;lt;");
});
it("null et undefined donnent une chaîne vide, pas 'null'", () => {
expect(P.escapeHtml(null)).toBe("");
expect(P.escapeHtml(undefined)).toBe("");
});
it("laisse le texte normal intact", () => {
expect(P.escapeHtml("Mötley Crüe")).toBe("Mötley Crüe");
});
});
describe("norm", () => {
it.each([
[null, ""], [undefined, ""], ["", ""], [" Oslo ", "Oslo"], [42, "42"],
])("norm(%s) === %s", (input, expected) => {
expect(P.norm(input)).toBe(expected);
});
// 0 est une valeur légitime : ne doit pas être avalé comme falsy.
it("conserve le zéro", () => {
expect(P.norm(0)).toBe("0");
});
});
describe("parseYear", () => {
it("accepte un nombre", () => {
expect(P.parseYear(1991)).toBe(1991);
});
it("extrait une année d'un texte libre", () => {
expect(P.parseYear("formé en 1991")).toBe(1991);
expect(P.parseYear("1984-1993")).toBe(1984);
});
it("ne reconnaît que les siècles plausibles", () => {
expect(P.parseYear("an 1234")).toBeNull();
expect(P.parseYear("2015")).toBe(2015);
});
it("renvoie null pour null/undefined", () => {
expect(P.parseYear(null)).toBeNull();
expect(P.parseYear(undefined)).toBeNull();
});
it("renvoie null pour un texte sans année", () => {
expect(P.parseYear("inconnu")).toBeNull();
});
});
describe("splitThemes", () => {
it("découpe sur virgule, point-virgule et slash", () => {
expect(P.splitThemes("Death, Satan; War/Occult"))
.toEqual(["Death", "Satan", "War", "Occult"]);
});
it("accepte déjà un tableau", () => {
expect(P.splitThemes([" a ", "", "b"])).toEqual(["a", "b"]);
});
it("élimine les fragments vides", () => {
expect(P.splitThemes("a,,;/b")).toEqual(["a", "b"]);
});
it("valeur vide → tableau vide", () => {
expect(P.splitThemes("")).toEqual([]);
expect(P.splitThemes(null)).toEqual([]);
});
});
describe("matchesQuery", () => {
it("une requête vide laisse tout passer", () => {
expect(P.matchesQuery(band(), "")).toBe(true);
});
it.each([
["nom", "mayhem"], ["genre", "black"], ["statut", "active"],
["pays", "no"], ["lieu", "oslo"],
])("cherche dans le %s", (_champ, q) => {
expect(P.matchesQuery(band(), q)).toBe(true);
});
it("cherche dans les thèmes", () => {
expect(P.matchesQuery(band({ themes: ["Satanism", "Death"] }), "satan")).toBe(true);
});
it("ne matche pas ce qui est absent", () => {
expect(P.matchesQuery(band(), "burzum")).toBe(false);
});
it("est insensible à la casse", () => {
expect(P.matchesQuery(band({ name: "MAYHEM" }), "mayhem")).toBe(true);
});
it("ne plante pas sur des champs manquants", () => {
expect(P.matchesQuery({ name: "X" }, "x")).toBe(true);
expect(P.matchesQuery({}, "zzz")).toBe(false);
});
});
describe("matchesFacet — logique tri-état", () => {
it("une facette absente de la Map passe (facette inconnue ≠ décochée)", () => {
expect(P.matchesFacet("Active", new Map(), "Unknown")).toBe(true);
});
it("une facette cochée passe", () => {
expect(P.matchesFacet("Active", new Map([["Active", true]]), "Unknown")).toBe(true);
});
it("une facette décochée ne passe pas", () => {
expect(P.matchesFacet("Active", new Map([["Active", false]]), "Unknown")).toBe(false);
});
it("une valeur vide retombe sur la clé de repli", () => {
const map = new Map([["Unknown", false]]);
expect(P.matchesFacet("", map, "Unknown")).toBe(false);
expect(P.matchesFacet(null, map, "Unknown")).toBe(false);
expect(P.matchesFacet(" ", map, "Unknown")).toBe(false);
});
it("la clé de repli cochée laisse passer les valeurs vides", () => {
expect(P.matchesFacet(null, new Map([["Unknown", true]]), "Unknown")).toBe(true);
});
it("les espaces autour de la valeur sont ignorés", () => {
expect(P.matchesFacet(" Active ", new Map([["Active", false]]), "Unknown")).toBe(false);
});
});
describe("matchesMacroGenre", () => {
const MACROS = [
{ key: "Black", terms: ["black"] },
{ key: "Doom", terms: ["doom", "stoner"] },
];
it("aucun macro-genre actif → tout passe", () => {
expect(P.matchesMacroGenre(band(), MACROS, new Map(), 0)).toBe(true);
});
it("matche sur un terme du macro-genre actif", () => {
const enabled = new Map([["Black", true]]);
expect(P.matchesMacroGenre(band({ genre: "Black Metal" }), MACROS, enabled, 1)).toBe(true);
});
it("matche n'importe lequel des termes d'un macro-genre", () => {
const enabled = new Map([["Doom", true]]);
expect(P.matchesMacroGenre(band({ genre: "Stoner Rock" }), MACROS, enabled, 1)).toBe(true);
});
it("ne matche pas un macro-genre inactif", () => {
const enabled = new Map([["Doom", true], ["Black", false]]);
expect(P.matchesMacroGenre(band({ genre: "Black Metal" }), MACROS, enabled, 1)).toBe(false);
});
// Quand un filtre de genre est actif, un groupe sans genre ne peut pas y
// répondre : il doit disparaître, pas passer par défaut.
it("un groupe sans genre est exclu dès qu'un macro-genre est actif", () => {
const enabled = new Map([["Black", true]]);
expect(P.matchesMacroGenre(band({ genre: "" }), MACROS, enabled, 1)).toBe(false);
expect(P.matchesMacroGenre(band({ genre: null }), MACROS, enabled, 1)).toBe(false);
});
it("est insensible à la casse du genre", () => {
const enabled = new Map([["Black", true]]);
expect(P.matchesMacroGenre(band({ genre: "BLACK METAL" }), MACROS, enabled, 1)).toBe(true);
});
});
describe("matchesTheme", () => {
it("aucun thème connu → tout passe", () => {
expect(P.matchesTheme(band(), new Map())).toBe(true);
});
it("tous les thèmes cochés → tout passe, même sans thème", () => {
const all = new Map([["War", true], ["Death", true]]);
expect(P.matchesTheme(band({ themes: [] }), all)).toBe(true);
});
it("filtre actif : un groupe sans thème est exclu", () => {
const partial = new Map([["War", true], ["Death", false]]);
expect(P.matchesTheme(band({ themes: [] }), partial)).toBe(false);
});
it("filtre actif : garde les groupes ayant au moins un thème coché", () => {
const partial = new Map([["War", true], ["Death", false]]);
expect(P.matchesTheme(band({ themes: ["War"] }), partial)).toBe(true);
expect(P.matchesTheme(band({ themes: ["Death"] }), partial)).toBe(false);
});
});
describe("matchesYear", () => {
const range = { min: 1970, max: 2020 };
it("aucun filtre posé → tout passe", () => {
expect(P.matchesYear(band(), { min: null, max: null }, range)).toBe(true);
});
it("plage non initialisée → tout passe", () => {
expect(P.matchesYear(band(), { min: 1990, max: 2000 }, { min: null, max: null })).toBe(true);
});
// Plage complète = pas de filtrage : les groupes sans année restent visibles.
it("plage complète → tout passe, y compris sans année", () => {
expect(P.matchesYear(band({ formed_year: null }), { ...range }, range)).toBe(true);
});
it("plage resserrée : garde ce qui est dedans", () => {
const f = { min: 1980, max: 1990 };
expect(P.matchesYear(band({ formed_year: 1984 }), f, range)).toBe(true);
expect(P.matchesYear(band({ formed_year: 1980 }), f, range)).toBe(true);
expect(P.matchesYear(band({ formed_year: 1990 }), f, range)).toBe(true);
});
it("plage resserrée : exclut ce qui est dehors", () => {
const f = { min: 1980, max: 1990 };
expect(P.matchesYear(band({ formed_year: 1979 }), f, range)).toBe(false);
expect(P.matchesYear(band({ formed_year: 1991 }), f, range)).toBe(false);
});
it("plage resserrée : exclut les groupes sans année", () => {
const f = { min: 1980, max: 1990 };
expect(P.matchesYear(band({ formed_year: null }), f, range)).toBe(false);
expect(P.matchesYear(band({ formed_year: undefined }), f, range)).toBe(false);
});
});
describe("keyFromLatLon", () => {
it("produit la même clé pour des coordonnées identiques", () => {
expect(P.keyFromLatLon(59.9139, 10.7522)).toBe(P.keyFromLatLon("59.9139", "10.7522"));
});
it("distingue des coordonnées différentes", () => {
expect(P.keyFromLatLon(59.9139, 10.7522)).not.toBe(P.keyFromLatLon(59.9140, 10.7522));
});
it("normalise à 6 décimales", () => {
expect(P.keyFromLatLon(1, 2)).toBe("1.000000,2.000000");
expect(P.keyFromLatLon(1.00000049, 2)).toBe("1.000000,2.000000");
});
});
describe("sortBands", () => {
const bands = [
band({ name: "Zyklon", genre: "Death", country: "NO", status: "Split-up", formed_year: 1998 }),
band({ name: "abbath", genre: "Black", country: "DE", status: "Active", formed_year: 2005 }),
band({ name: "Mayhem", genre: "Black", country: "NO", status: "Active", formed_year: 1984 }),
];
it("ne modifie pas le tableau d'entrée", () => {
const copy = [...bands];
P.sortBands(bands, "az");
expect(bands).toEqual(copy);
});
it("tri alphabétique insensible à la casse et aux accents", () => {
expect(P.sortBands(bands, "az").map((b) => b.name)).toEqual(["abbath", "Mayhem", "Zyklon"]);
});
it("mode inconnu retombe sur le tri alphabétique", () => {
expect(P.sortBands(bands, "nimportequoi").map((b) => b.name))
.toEqual(["abbath", "Mayhem", "Zyklon"]);
});
it("tri par année, décroissant", () => {
expect(P.sortBands(bands, "year").map((b) => b.formed_year)).toEqual([2005, 1998, 1984]);
});
it("les groupes sans année finissent en dernier", () => {
const withNull = [...bands, band({ name: "Sans", formed_year: null })];
expect(P.sortBands(withNull, "year").at(-1).name).toBe("Sans");
});
it.each(["status", "genre", "country"])("tri par %s, départagé par le nom", (mode) => {
const sorted = P.sortBands(bands, mode);
for (let i = 1; i < sorted.length; i++) {
const prev = (sorted[i - 1][mode] || "").toLowerCase();
const cur = (sorted[i][mode] || "").toLowerCase();
expect(prev <= cur).toBe(true);
}
});
it("départage bien à valeur égale", () => {
// abbath et Mayhem sont tous deux Black/Active → départagés par nom
expect(P.sortBands(bands, "genre").map((b) => b.name)).toEqual(["abbath", "Mayhem", "Zyklon"]);
});
it("liste vide", () => {
expect(P.sortBands([], "az")).toEqual([]);
});
});

103
eslint.config.js Normal file
View file

@ -0,0 +1,103 @@
import js from "@eslint/js";
import globals from "globals";
/**
* Config ESLint « flat » (ESLint 9).
*
* Trois environnements coexistent dans ce repo et n'ont pas les mêmes globales :
* - apps/api : Node ESM
* - apps/web/site : scripts classiques navigateur (Leaflet en global)
* - apps/admin/site : scripts classiques navigateur
*/
export default [
{
ignores: [
"**/node_modules/**",
"apps/web/quizz-site/**",
"reports/**",
".stryker-tmp/**",
"coverage/**",
],
},
js.configs.recommended,
// ---- API : Node, ESM ----
{
files: ["apps/api/src/**/*.js"],
languageOptions: {
ecmaVersion: 2023,
sourceType: "module",
globals: { ...globals.node },
},
rules: {
"no-unused-vars": ["error", { argsIgnorePattern: "^_", caughtErrors: "none" }],
"no-console": "off",
eqeqeq: ["error", "smart"],
"prefer-const": "error",
"no-var": "error",
"no-throw-literal": "error",
"require-await": "warn",
},
},
// ---- Tests (vitest). Certains tournent sous jsdom → globales navigateur aussi. ----
{
files: ["apps/*/test/**/*.js"],
languageOptions: {
ecmaVersion: 2023,
sourceType: "module",
globals: { ...globals.node, ...globals.browser },
},
rules: {
"no-unused-vars": ["error", { argsIgnorePattern: "^_", caughtErrors: "none" }],
},
},
// ---- Frontends : scripts classiques navigateur ----
{
files: ["apps/web/site/**/*.js", "apps/admin/site/**/*.js"],
languageOptions: {
ecmaVersion: 2023,
sourceType: "script",
globals: {
...globals.browser,
// Chargés via <script> (CDN ou fichier local) avant app.js
L: "readonly",
LOCALES: "readonly",
noUiSlider: "readonly",
BMPure: "readonly",
// Footer CJS des modules « purs » partagés navigateur/test
module: "writable",
},
},
rules: {
"no-unused-vars": ["error", { argsIgnorePattern: "^_", caughtErrors: "none" }],
// `try { … } catch {}` volontaire sur les appels best-effort (logout, télémétrie)
"no-empty": ["error", { allowEmptyCatch: true }],
"no-console": ["warn", { allow: ["warn", "error"] }],
eqeqeq: ["error", "smart"],
"prefer-const": "error",
"no-var": "error",
// Les frontends déclarent volontairement des globales partagées entre fichiers
"no-redeclare": ["error", { builtinGlobals: false }],
},
},
// ---- Traductions : les espaces insécables fines sont de la typographie
// volontaire (français, tchèque…), pas des caractères parasites. ----
{
files: ["apps/web/site/locales.js"],
rules: { "no-irregular-whitespace": "off" },
},
// ---- Config outillage à la racine ----
{
files: ["*.js", "*.config.js"],
languageOptions: {
ecmaVersion: 2023,
sourceType: "module",
globals: { ...globals.node },
},
},
];

View file

@ -1,11 +1,49 @@
# URL complète vers la DB PostgreSQL (gérée séparément par Coolify) # ------------------------------------------------------------------
# Base de données (gérée séparément par Coolify, standalone)
# ------------------------------------------------------------------
DATABASE_URL=postgresql://bm:PASSWORD@HOST:5432/bm DATABASE_URL=postgresql://bm:PASSWORD@HOST:5432/bm
NOMINATIM_EMAIL= # ------------------------------------------------------------------
NOMINATIM_USER_AGENT=bm-map/0.1 # API — authentification
# ------------------------------------------------------------------
# Jeton Bearer des endpoints d'import machine-à-machine (/admin/import).
BM_IMPORT_TOKEN= BM_IMPORT_TOKEN=
# OBLIGATOIRE : secret de signature des sessions du dashboard admin.
# Minimum 32 caractères, sinon TOUTE connexion admin échoue en 500.
# openssl rand -hex 32
ADMIN_JWT_SECRET=
# Compte admin créé au démarrage s'il n'existe pas (les deux sont requis,
# sinon aucun compte n'est créé et le dashboard est inaccessible).
# Générer le hash :
# node -e "console.log(require('bcryptjs').hashSync(process.argv[1],12))" 'mon-mot-de-passe'
ADMIN_SEED_USERNAME=
ADMIN_SEED_PASSWORD_HASH=
# Origines autorisées en CORS, séparées par des virgules.
# dev : https://dev.metalfrom.eu
# prod : https://metalfrom.eu,https://www.metalfrom.eu
CORS_ORIGINS=
# ------------------------------------------------------------------
# Géocodage
# ------------------------------------------------------------------
# Fournisseur de géocodage (pipeline actuel).
GEOAPIFY_API_KEY=
# Désambiguïsation LLM des lieux que Geoapify ne résout pas.
GROQ_API_KEY=
# ------------------------------------------------------------------
# pgAdmin
# ------------------------------------------------------------------
PGADMIN_EMAIL= PGADMIN_EMAIL=
PGADMIN_PASSWORD= PGADMIN_PASSWORD=
# ------------------------------------------------------------------
# Divers
# ------------------------------------------------------------------
TZ=Europe/Paris TZ=Europe/Paris
PUID=1001 PUID=1001
PGID=1001 PGID=1001

20
jsconfig.json Normal file
View file

@ -0,0 +1,20 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "Bundler",
"lib": ["ES2022", "DOM"],
"allowJs": true,
"checkJs": true,
"noEmit": true,
"strict": false,
"noImplicitAny": false,
"strictNullChecks": false,
"skipLibCheck": true,
"resolveJsonModule": true,
"types": ["node"],
"maxNodeModuleJsDepth": 0
},
"include": ["apps/api/src/**/*.js", "apps/api/test/**/*.js"],
"exclude": ["node_modules", "**/node_modules"]
}

6728
package-lock.json generated Normal file

File diff suppressed because it is too large Load diff

43
package.json Normal file
View file

@ -0,0 +1,43 @@
{
"name": "metalfromeu",
"version": "0.0.1",
"private": true,
"type": "module",
"description": "Monorepo metalfrom.eu — outillage qualité (lint, types, tests). Chaque app garde son propre runtime.",
"workspaces": [
"apps/api"
],
"scripts": {
"lint": "eslint .",
"lint:fix": "eslint . --fix",
"typecheck": "tsc -p jsconfig.json",
"test": "vitest run",
"test:watch": "vitest",
"test:cov": "vitest run --coverage",
"test:mutation": "stryker run",
"test:mutation:full": "stryker run stryker.full.config.json",
"lint:py": "ruff check .",
"lint:py:fix": "ruff check . --fix",
"test:py": "npm run test:py:geocoder && npm run test:py:crawler",
"test:py:geocoder": "cd apps/geocoder && python -m pytest tests -q",
"test:py:crawler": "cd apps/crawler && python -m pytest tests -q",
"audit:js": "npm audit --audit-level=high",
"audit:py": "pip-audit -r apps/api/../crawler/requirements.txt -r apps/geocoder/requirements.txt",
"_comment_check": "`check` = la porte de qualité lancée avant chaque push (~30s). Les tests de mutation et les audits de dépendances en sont exclus : trop lents / dépendants du réseau. Lancer `npm run check:full` avant une release.",
"check": "npm run lint && npm run lint:py && npm run typecheck && npm run test && npm run test:py",
"check:full": "npm run check && npm run audit:js && npm run audit:py && npm run test:mutation:full",
"hooks:install": "git config core.hooksPath .githooks"
},
"devDependencies": {
"@eslint/js": "^9.17.0",
"@stryker-mutator/core": "^8.7.1",
"@stryker-mutator/vitest-runner": "^8.7.1",
"@vitest/coverage-v8": "^2.1.8",
"axe-core": "^4.10.2",
"eslint": "^9.17.0",
"globals": "^15.14.0",
"jsdom": "^25.0.1",
"typescript": "^5.7.2",
"vitest": "^2.1.8"
}
}

41
pyproject.toml Normal file
View file

@ -0,0 +1,41 @@
[tool.ruff]
# Linting Python (équivalent d'ESLint côté crawler/geocoder).
line-length = 110
target-version = "py312"
exclude = [".git", "node_modules", "__pycache__"]
[tool.ruff.lint]
select = [
"E", # pycodestyle
"F", # pyflakes — imports/variables inutilisés, noms indéfinis
"I", # isort — ordre des imports
"B", # bugbear — pièges classiques (mutable default, boucle+except…)
"UP", # pyupgrade
"S", # bandit — analyse de sécurité (SQLi par f-string, subprocess, secrets…)
"C4", # comprehensions
]
ignore = [
"E501", # longueur gérée par le formateur
"S101", # `assert` : légitime dans les tests
]
[tool.ruff.lint.per-file-ignores]
# Les requêtes SQL sont construites avec des placeholders psycopg2 (%s) ;
# les rares f-strings ne portent que des noms de colonnes issus d'allowlists.
"apps/crawler/src/db.py" = ["S608"]
"apps/geocoder/src/*.py" = ["S608"]
"**/tests/*.py" = ["S105", "S106"]
# `random` sert au jitter de politesse entre deux requêtes HTTP, pas à générer
# des secrets : S311 (générateur non cryptographique) ne s'applique pas.
"apps/crawler/src/polite.py" = ["S311"]
"apps/geocoder/src/worker.py" = ["S311", "S608"]
# md5 sert d'empreinte de contenu pour détecter les pages inchangées,
# jamais de primitive de sécurité.
"apps/crawler/src/scraper_band.py" = ["S324"]
[tool.pytest.ini_options]
# Chaque app est testée depuis sa propre racine (`rootdir`), d'où le chemin relatif
# `src.` dans les imports de test. Voir le script `test:py` du package.json.
testpaths = ["tests"]
python_files = ["test_*.py"]
addopts = "-q --strict-markers"

9
requirements-dev.txt Normal file
View file

@ -0,0 +1,9 @@
# Outillage qualité Python (hors runtime des apps).
# pip install -r requirements-dev.txt
ruff==0.8.4
pytest==8.3.4
pip-audit==2.7.3
# Les tests importent src.db, qui importe psycopg2 au chargement du module.
# Aucune connexion n'est ouverte : get_conn est remplacé dans les tests.
psycopg2-binary==2.9.9

41
stryker.config.json Normal file
View file

@ -0,0 +1,41 @@
{
"$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
"_comment_role": "Tests de mutation : la seule mesure qui prouve qu'un test ÉCHOUE quand le code casse. La couverture de ligne ne dit que 'ce code a été exécuté', pas 'ce comportement est vérifié'. NE FAIT PAS partie de `npm run check` (cf. README-CI.md) : c'est un audit périodique, pas une porte de pré-push.",
"_comment_perf": "Trois réglages font l'essentiel du temps d'exécution : (1) concurrency alignée sur les cœurs disponibles, (2) coverageAnalysis perTest qui ne rejoue que les tests couvrant chaque mutant, (3) exclusion des mutants StringLiteral sur le code SQL — voir _comment_mutators.",
"_comment_mutators": "excludedMutations StringLiteral : adminRoutes.js et app.js sont à ~80 % des requêtes SQL en template literals. Muter le CONTENU d'une chaîne SQL produit des mutants soit triviaux (la requête ne compile plus), soit indétectables — dans les deux cas ils ne mesurent rien et représentaient la majorité des 1942 mutants. Ce qui compte réellement dans un handler de route, ce sont les CONDITIONS (gardes d'auth, bornes, branches d'erreur), qui restent toutes mutées.",
"packageManager": "npm",
"testRunner": "vitest",
"vitest": {
"configFile": "vitest.mutation.config.js"
},
"reporters": [
"progress",
"clear-text",
"html"
],
"htmlReporter": {
"fileName": "reports/mutation/index.html"
},
"mutate": [
"apps/api/src/validate.js",
"apps/api/src/adminAuth.js"
],
"mutator": {
"excludedMutations": [
"StringLiteral"
]
},
"coverageAnalysis": "perTest",
"ignoreStatic": true,
"concurrency": 10,
"timeoutMS": 15000,
"_comment_incremental": "Après un premier run complet, seuls les fichiers modifiés (et leurs tests) sont rejoués. Les runs suivants tombent de quelques minutes à quelques secondes. Supprimer reports/stryker-incremental.json force un run complet.",
"incremental": true,
"incrementalFile": "reports/stryker-incremental.json",
"thresholds": {
"high": 90,
"low": 75,
"break": 70
},
"tempDirName": ".stryker-tmp"
}

43
stryker.full.config.json Normal file
View file

@ -0,0 +1,43 @@
{
"$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
"_comment_role": "Tests de mutation : la seule mesure qui prouve qu'un test ÉCHOUE quand le code casse. La couverture de ligne ne dit que 'ce code a été exécuté', pas 'ce comportement est vérifié'. NE FAIT PAS partie de `npm run check` (cf. README-CI.md) : c'est un audit périodique, pas une porte de pré-push.",
"_comment_perf": "Trois réglages font l'essentiel du temps d'exécution : (1) concurrency alignée sur les cœurs disponibles, (2) coverageAnalysis perTest qui ne rejoue que les tests couvrant chaque mutant, (3) exclusion des mutants StringLiteral sur le code SQL — voir _comment_mutators.",
"_comment_mutators": "excludedMutations StringLiteral : adminRoutes.js et app.js sont à ~80 % des requêtes SQL en template literals. Muter le CONTENU d'une chaîne SQL produit des mutants soit triviaux (la requête ne compile plus), soit indétectables — dans les deux cas ils ne mesurent rien et représentaient la majorité des 1942 mutants. Ce qui compte réellement dans un handler de route, ce sont les CONDITIONS (gardes d'auth, bornes, branches d'erreur), qui restent toutes mutées.",
"packageManager": "npm",
"testRunner": "vitest",
"vitest": {
"configFile": "vitest.mutation.full.config.js"
},
"reporters": [
"progress",
"clear-text",
"html"
],
"htmlReporter": {
"fileName": "reports/mutation-full/index.html"
},
"mutate": [
"apps/api/src/validate.js",
"apps/api/src/adminAuth.js",
"apps/api/src/adminRoutes.js",
"apps/api/src/app.js"
],
"mutator": {
"excludedMutations": [
"StringLiteral"
]
},
"coverageAnalysis": "perTest",
"ignoreStatic": true,
"concurrency": 10,
"timeoutMS": 15000,
"_comment_incremental": "Après un premier run complet, seuls les fichiers modifiés (et leurs tests) sont rejoués. Les runs suivants tombent de quelques minutes à quelques secondes. Supprimer reports/stryker-incremental.json force un run complet.",
"incremental": true,
"incrementalFile": "reports/stryker-incremental-full.json",
"thresholds": {
"high": 85,
"low": 60,
"break": 55
},
"tempDirName": ".stryker-tmp"
}

45
vitest.config.js Normal file
View file

@ -0,0 +1,45 @@
import { defineConfig } from "vitest/config";
/**
* Environnement `node` par défaut (démarrage quasi instantané). Les rares
* fichiers qui ont besoin du DOM déclarent leur propre environnement via un
* docblock en tête de fichier :
*
* // @vitest-environment jsdom
*
* Aucun test ne démarre de conteneur ni de vraie base : les routes API sont
* testées via `fastify.inject()` avec un faux pool (test/helpers/fakePool.js),
* et les tests a11y parsent le HTML statique avec jsdom + axe-core.
*/
export default defineConfig({
test: {
environment: "node",
// Pool laissé au défaut (forks) : mesuré contre les alternatives sur cette
// suite, 401 tests —
// forks (défaut) : 3,5 s
// threads, isolate: true : lent
// threads, isolate: false : 18,8 s (mise en place de jsdom pénalisée)
// L'intuition « threads sans isolation = plus rapide » est fausse ici.
// Re-mesurer avant de changer.
include: ["apps/*/test/**/*.test.js"],
reporters: ["default"],
coverage: {
provider: "v8",
reporter: ["text", "lcov"],
include: ["apps/api/src/**/*.js"],
// migrate.js s'exécute au démarrage du conteneur et appelle process.exit :
// le couvrir demanderait une vraie base, ce qu'on s'interdit ici.
// server.js n'est qu'un point d'entrée (5 lignes), la logique est dans app.js.
exclude: ["apps/api/src/migrate.js", "apps/api/src/server.js"],
// Seuils = cliquet anti-régression. Ils ne prouvent PAS que les tests
// vérifient quelque chose (seule la mutation le fait, cf. `npm run
// test:mutation`), mais ils empêchent d'ajouter du code non exécuté.
thresholds: {
statements: 85,
branches: 80,
functions: 85,
lines: 85,
},
},
},
});

23
vitest.mutation.config.js Normal file
View file

@ -0,0 +1,23 @@
import { defineConfig } from "vitest/config";
/**
* Configuration Vitest dédiée aux tests de mutation.
*
* Stryker recharge l'ensemble des fichiers de test à chaque activation de
* mutant. Avec les 11 fichiers de la suite complète, cet amorçage représentait
* l'essentiel du temps d'exécution (~270 ms par mutant, alors qu'un test met
* ~8 ms). On ne déclare donc ici que les fichiers qui couvrent réellement les
* modules mutés les autres ne feraient qu'être chargés puis ignorés.
*
* À tenir synchronisé avec le champ `mutate` de stryker.config.json.
*/
export default defineConfig({
test: {
environment: "node",
include: [
"apps/api/test/validate.test.js",
"apps/api/test/adminAuth.test.js",
],
reporters: ["default"],
},
});

View file

@ -0,0 +1,18 @@
import { defineConfig } from "vitest/config";
/**
* Config Vitest du profil de mutation COMPLET (`npm run test:mutation:full`).
*
* Même principe que vitest.mutation.config.js : ne charger que les fichiers qui
* couvrent le code muté. Ici tout le code API est muté, donc tous les tests API
* sont nécessaires mais surtout PAS les tests frontend, dont le fichier a11y
* initialise jsdom (~2 s) à chaque activation de mutant alors qu'il ne couvre
* aucune ligne de l'API.
*/
export default defineConfig({
test: {
environment: "node",
include: ["apps/api/test/**/*.test.js"],
reporters: ["default"],
},
});