Implementation initiale : calcul de volume 2.5D CloudCompare et test de robustesse par decimation spatiale progressive
- Backend NestJS orchestrant CloudCompare CLI (stats, decimation x2 sur 5 etapes, volume, assemblage .bin) - Scripts Python (laspy, scipy, matplotlib) pour la conversion LAS/LAZ et la generation d'images - Frontend web simple (Tailwind CDN) avec historique des runs - Image Docker (debian:trixie-slim + cloudcompare apt + xvfb) prete pour deploiement Coolify - Lint/typecheck/tests unitaires integres comme portes de qualite au build Docker - CLAUDE.md documentant les comportements CloudCompare CLI verifies empiriquement
This commit is contained in:
commit
79daa30a8a
36 changed files with 12329 additions and 0 deletions
10
.dockerignore
Normal file
10
.dockerignore
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
node_modules
|
||||||
|
dist
|
||||||
|
data
|
||||||
|
*.log
|
||||||
|
.git
|
||||||
|
.gitignore
|
||||||
|
.claude
|
||||||
|
coverage
|
||||||
|
README.md
|
||||||
|
CLAUDE.md
|
||||||
2
.gitattributes
vendored
Normal file
2
.gitattributes
vendored
Normal file
|
|
@ -0,0 +1,2 @@
|
||||||
|
* text=auto eol=lf
|
||||||
|
*.sh text eol=lf
|
||||||
7
.gitignore
vendored
Normal file
7
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
node_modules/
|
||||||
|
dist/
|
||||||
|
data/
|
||||||
|
*.log
|
||||||
|
.DS_Store
|
||||||
|
coverage/
|
||||||
|
.claude/
|
||||||
170
CLAUDE.md
Normal file
170
CLAUDE.md
Normal file
|
|
@ -0,0 +1,170 @@
|
||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
Notes pour toute future session travaillant sur ce repo. Ce fichier documente des faits
|
||||||
|
verifies empiriquement sur CloudCompare 2.13.2 (Windows officiel ET paquet apt Debian trixie) —
|
||||||
|
pas de la documentation officielle recopiee, mais des comportements observes directement, souvent
|
||||||
|
non documentes ou contredisant la doc/le wiki.
|
||||||
|
|
||||||
|
## Contexte du projet
|
||||||
|
|
||||||
|
Pipeline : nuage de points (LAS/LAZ/COPC.laz/BIN) -> calcul de volume 2.5D CloudCompare -> decimation
|
||||||
|
spatiale progressive x2 sur 5 etapes (chainee, chaque niveau decime depuis le precedent, pas depuis
|
||||||
|
l'original) -> volume recalcule a chaque niveau -> assemblage des 6 nuages dans un seul `.bin` ->
|
||||||
|
rapport CSV/JSON + images. UI web (NestJS + Tailwind CDN) avec historique des runs (SQLite).
|
||||||
|
E57 volontairement hors perimetre.
|
||||||
|
|
||||||
|
## CloudCompare CLI — comportements verifies
|
||||||
|
|
||||||
|
### `-VOLUME` (calcul de volume 2.5D)
|
||||||
|
|
||||||
|
- Sur un nuage **unique**, utiliser `-CONST_HEIGHT <valeur>` comme plan de reference (le nuage charge
|
||||||
|
est le "ceiling", le plan constant est le "ground"). Sans `-CONST_HEIGHT`, la commande attend DEUX
|
||||||
|
nuages charges.
|
||||||
|
- Syntaxe complete : `-VOLUME -GRID_STEP <val> [-VERT_DIR 0/1/2] [-CONST_HEIGHT <val>] [-GROUND_IS_FIRST] [-OUTPUT_MESH]`.
|
||||||
|
- **Piege majeur non documente** : le rapport texte `VolumeCalculationReport_<date>.txt` n'est genere
|
||||||
|
QUE si `-AUTO_SAVE ON` (le defaut). Avec `-AUTO_SAVE OFF` (qu'on utilise partout ailleurs pour
|
||||||
|
eviter de polluer le dossier de travail avec des fichiers intermediaires), la commande `-VOLUME`
|
||||||
|
s'execute (log "[2.5D VOLUME CALCULATION] finished") mais **aucun rapport n'est ecrit, aucune
|
||||||
|
erreur n'est loggee**. Il faut explicitement repasser `-AUTO_SAVE ON` juste pour l'appel `-VOLUME`.
|
||||||
|
Voir `CcRunnerService.run(args, cwd, { autoSave: true })`.
|
||||||
|
- Avec `-AUTO_SAVE ON`, `-VOLUME` sauvegarde AUSSI automatiquement une grille de difference de hauteur
|
||||||
|
nommee `<nom>_HEIGHT_DIFFERENCE_<date>.bin` qu'on ne veut pas garder — on la detecte (meme technique
|
||||||
|
que pour le rapport, cf ci-dessous) et on la supprime juste apres (voir pipeline.service.ts).
|
||||||
|
- **CloudCompare ecrit ses fichiers de sortie automatiques (rapport, grilles) relativement au dossier
|
||||||
|
du fichier charge via `-O`, pas au `cwd` du process.** Si on charge `-O levels/L0_full.bin`, le
|
||||||
|
rapport atterrit dans `levels/VolumeCalculationReport_*.txt`, pas a la racine du `cwd`. D'ou la
|
||||||
|
necessite d'une recherche **recursive** (`CcRunnerService.findLatestFile`) plutot qu'un simple
|
||||||
|
`readdir` du dossier de travail.
|
||||||
|
- Format du rapport texte (verifie, stable) — cle:valeur, une ligne par info, parsable par regex :
|
||||||
|
```
|
||||||
|
Volume: 7.834176
|
||||||
|
Surface: 100.774998
|
||||||
|
----------------------
|
||||||
|
Added volume: (+)7.975798
|
||||||
|
Removed volume: (-)0.141622
|
||||||
|
----------------------
|
||||||
|
Matching cells: 99.8%
|
||||||
|
Non-matching cells:
|
||||||
|
ground = 0.2%
|
||||||
|
ceil = 0.0%
|
||||||
|
Average neighbors per cell: 8.0 / 8.0
|
||||||
|
```
|
||||||
|
- **`Matching cells: X%`** est le meilleur signal natif de robustesse : c'est le % de cellules de la
|
||||||
|
grille de calcul ou les deux surfaces sont effectivement comparables. Il s'effondre des que la
|
||||||
|
decimation produit un nuage plus clairseme que la resolution de grille utilisee pour le calcul de
|
||||||
|
volume — exactement l'indicateur cherche pour ce projet (voir `matchingCellsWarnThreshold` dans
|
||||||
|
`config.ts`, defaut 90%).
|
||||||
|
- Le `-GRID_STEP` du calcul de volume doit rester **fixe** (le meme pour les 6 niveaux) pour que les
|
||||||
|
volumes restent comparables entre eux — sinon on mele l'effet de la decimation avec l'effet d'un
|
||||||
|
changement de resolution de calcul. On le derive une seule fois du pas spatial initial (voir plus
|
||||||
|
bas) et on le reutilise identique partout.
|
||||||
|
|
||||||
|
### `-SS SPATIAL` / `-SS RANDOM` (sous-echantillonnage)
|
||||||
|
|
||||||
|
- `-SS SPATIAL <distance>` : distance minimale entre points, PAS un ratio de points cible. Le nombre
|
||||||
|
de points obtenu depend de la densite reelle du nuage — jamais garanti a un pourcentage exact.
|
||||||
|
- `-SS RANDOM <n>` : sous-echantillonne a `n` points. Si `n` >= nombre de points du nuage, ne plante
|
||||||
|
pas (comportement observe : garde le nuage tel quel).
|
||||||
|
- Log de sortie a parser : `[SUBSAMPLE]` puis `Result: <N> points` (regex utilisee :
|
||||||
|
`/\[SUBSAMPLE\][\s\S]*?Result:\s*(\d+)\s*points/`).
|
||||||
|
- `-AUTO_SAVE OFF` fonctionne correctement ici (contrairement a `-VOLUME`) : pas de fichier auto-sauve
|
||||||
|
parasite, seul `-SAVE_CLOUDS FILE "nom.bin"` explicite produit un fichier, avec le nom voulu.
|
||||||
|
|
||||||
|
### `-SAVE_CLOUDS`
|
||||||
|
|
||||||
|
- `-SAVE_CLOUDS FILE "nom.ext"` : nom de sortie explicite (fonctionne, contrairement a ce que
|
||||||
|
certains threads du forum CloudCompare laissent penser).
|
||||||
|
- `-SAVE_CLOUDS ALL_AT_ONCE FILE "nom.bin"` avec **plusieurs nuages charges** (plusieurs `-O`) sans
|
||||||
|
`-MERGE_CLOUDS` : sauvegarde tous les nuages comme entites **distinctes** dans un seul `.bin` (le
|
||||||
|
format `.bin` de CloudCompare supporte nativement une hierarchie de plusieurs nuages). Verifie en
|
||||||
|
rouvrant le fichier : chaque nuage retrouve son point count exact, sans fusion de geometrie. C'est
|
||||||
|
la technique utilisee pour assembler les 6 niveaux de decimation dans un seul fichier de sortie.
|
||||||
|
**`-MERGE_CLOUDS` fusionnerait la geometrie en un seul nuage — a ne PAS utiliser ici.**
|
||||||
|
- Pas de commande `-RENAME_CLOUDS` (n'existe pas, testee = "Unknown or misplaced command"). Le nom de
|
||||||
|
chaque nuage dans le `.bin` fusionne est derive du nom de fichier source charge — on nomme donc les
|
||||||
|
fichiers intermediaires de facon descriptive (`L0_full.bin`, `L1_step0.05.bin`, ...) plutot que de
|
||||||
|
chercher a renommer les entites apres coup.
|
||||||
|
|
||||||
|
### Autres commandes
|
||||||
|
|
||||||
|
- `-RASTERIZE -GRID_STEP <val> -OUTPUT_RASTER_Z` : export GeoTIFF. **Fonctionne sur le build Windows
|
||||||
|
officiel, mais PLANTE (assertion `false` dans `ccRasterizeTool.cpp:ExportGeoTiff`, core dump) sur le
|
||||||
|
paquet apt Debian trixie**, faute de support GDAL compile dans ce paquet. Ne pas utiliser cette
|
||||||
|
voie dans l'image Docker — voir section "Paquet apt Debian" ci-dessous pour la solution retenue.
|
||||||
|
- Les commandes s'appliquent comme une **machine a etats sequentielle sur UN SEUL process** : on ne
|
||||||
|
peut pas "reprendre" une session precedente. Chaque etape logique du pipeline (stats, conversion,
|
||||||
|
decimation, volume, export) = un appel CLI independant, avec son propre `-O` de rechargement.
|
||||||
|
- `-SILENT` doit etre le tout premier argument (ou juste apres `-VERBOSITY`).
|
||||||
|
- `-PREC <n>` controle la precision decimale des exports ASCII (`-C_EXPORT_FMT ASC`).
|
||||||
|
|
||||||
|
## Paquet apt Debian `cloudcompare` (trixie, 2.13.2) — limitations et contournements
|
||||||
|
|
||||||
|
Choix d'archi : `debian:trixie-slim` + `apt-get install cloudcompare` plutot qu'une compilation depuis
|
||||||
|
les sources (des heures de build, fragile). Debian trixie est la seule distro testee avec une version
|
||||||
|
recente (2.13.2) directement en apt `main` (Ubuntu jammy/noble n'ont que 2.11.3 en `universe`).
|
||||||
|
Mais ce paquet est **allege par rapport au build officiel Windows** :
|
||||||
|
|
||||||
|
1. **Pas de plugin LAS/LAZ** (`dpkg -L cloudcompare` ne montre que
|
||||||
|
`libQCORE_IO_PLUGIN.so`, aucun `QLAS_IO_PLUGIN`). Ouvrir un `.las`/`.laz` donne :
|
||||||
|
`[Load] Can't guess file format: unhandled file extension 'las'`.
|
||||||
|
**Solution retenue** : conversion LAS/LAZ/COPC -> ASCII XYZ via `laspy` (Python, `python/las_to_xyz.py`)
|
||||||
|
*avant* tout traitement CloudCompare. L'import ASCII XYZ, lui, fonctionne nativement sans aucun
|
||||||
|
plugin (verifie : `CloudCompare -O fichier.xyz` marche directement sur ce paquet). Un `.copc.laz`
|
||||||
|
est lu par laspy comme un LAZ standard (l'indexation octree COPC est ignoree, seuls les points
|
||||||
|
comptent ici).
|
||||||
|
2. **Export GeoTIFF cassé** (voir `-RASTERIZE` ci-dessus). **Solution retenue** : les cartes de hauteur
|
||||||
|
sont generees nous-memes (`python/render_images.py heightmap`) via export ASCII du nuage +
|
||||||
|
binning `scipy.stats.binned_statistic_2d` + rendu `matplotlib`, sans passer par le rasterizer
|
||||||
|
CloudCompare.
|
||||||
|
3. **CloudCompare reste une appli Qt, meme en `-SILENT`** : plante avec `QXcbConnection: Could not
|
||||||
|
connect to display` sans serveur X. Necessite `xvfb`. On demarre UN Xvfb persistant dans
|
||||||
|
`docker/entrypoint.sh` (pas un `xvfb-run` par appel CLI, plus rapide et evite les conflits de
|
||||||
|
lockfile entre appels concurrents).
|
||||||
|
4. Si un futur besoin necessite E57 : le meme probleme de plugin manquant se posera probablement
|
||||||
|
(a verifier — `QE57_IO_PLUGIN` n'apparaissait pas non plus dans `dpkg -L cloudcompare`).
|
||||||
|
|
||||||
|
**Si CloudCompare est mis a jour dans une future version du paquet Debian et regagne GDAL/LAS**, ces
|
||||||
|
contournements resteront fonctionnels sans rien casser (ils n'utilisent jamais les fonctionnalites
|
||||||
|
manquantes), mais pourraient etre simplifies.
|
||||||
|
|
||||||
|
## Pieges d'implementation generaux (au-dela de CloudCompare)
|
||||||
|
|
||||||
|
- **Chemins `/c/...` (style MSYS/git-bash) vs chemins Windows natifs** : un `python.exe` natif Windows
|
||||||
|
ne comprend PAS `/c/Users/...` — seulement `C:/Users/...` ou `C:\Users\...`. `curl`/`cat`/`ls`
|
||||||
|
(MSYS) acceptent les deux, mais un process Windows natif lance depuis bash avec un chemin `/c/...`
|
||||||
|
en argument echouera silencieusement (`FileNotFoundError`) si ce chemin est passe tel quel a une
|
||||||
|
fonction Python (`open(...)`) plutot que d'etre resolu par le shell. Toujours utiliser des chemins
|
||||||
|
style `C:/...` quand on passe un chemin en argument a un outil natif Windows depuis bash.
|
||||||
|
- **TypeScript + closures + `let` mutable** : `tsc` peut perdre le narrowing d'une variable `let`
|
||||||
|
mutee dans une closure asynchrone appelee avant un `return` conditionnel (erreur `Property 'x' does
|
||||||
|
not exist on type 'never'`). Contournement : accumuler dans un tableau (`push`) plutot que de muter
|
||||||
|
une variable `best` capturee, puis trier/reduire a la fin.
|
||||||
|
- **Nest CLI + fichiers de test** : `nest build` doit utiliser `tsconfig.build.json` (qui exclut
|
||||||
|
`**/*.spec.ts`) pour ne pas emettre les tests dans `dist/`. Le `tsconfig.json` de base (sans
|
||||||
|
exclude) reste utilise par `tsc --noEmit` (script `typecheck`) et par `ts-jest`, pour que les tests
|
||||||
|
soient bien type-checkes.
|
||||||
|
- **`better-sqlite3`** necessite des outils de build natifs (python3/make/g++) — presents par defaut
|
||||||
|
dans l'image `node:20-bookworm` utilisee pour le stage de build, absents volontairement du stage
|
||||||
|
final (juste `node_modules` deja compile est copie).
|
||||||
|
|
||||||
|
## Deploiement (Coolify)
|
||||||
|
|
||||||
|
- Image construite en deux etapes : build (Node, avec lint+typecheck+test+compilation) puis runtime
|
||||||
|
(Debian + CloudCompare + xvfb + Python). Le build Docker **echoue** si lint, typecheck ou tests
|
||||||
|
echouent (`RUN npm run lint|typecheck|test` avant `RUN npm run build` dans le Dockerfile) — sert de
|
||||||
|
garde-fou avant tout deploiement automatique via webhook.
|
||||||
|
- `docker-compose.yml` utilise un **volume nomme** (`app-data:/data`), pas un bind mount vers `./data` :
|
||||||
|
Coolify re-clone le repo a chaque deploiement, un bind mount relatif au checkout perdrait les
|
||||||
|
donnees (uploads, resultats, base sqlite) a chaque redeploy.
|
||||||
|
- `HEALTHCHECK` dans le Dockerfile (`curl` sur `/`) : Coolify l'utilise pour determiner si le
|
||||||
|
deploiement a reussi.
|
||||||
|
- Le port d'ecoute est configurable via `PORT` (`config.ts` lit `process.env.PORT`, defaut 3000).
|
||||||
|
|
||||||
|
## Constantes du pipeline (voir `src/config.ts`)
|
||||||
|
|
||||||
|
- `decimationFactor = 2`, `decimationSteps = 5` (fixes par la spec, pas exposes en config utilisateur).
|
||||||
|
- `gridStepMultiplier = 2` : le grid step de `-VOLUME` = pas spatial initial x2.
|
||||||
|
- `statsSampleCap = 50000` : nb de points echantillonnes pour estimer la distance mediane au plus
|
||||||
|
proche voisin (KD-tree scipy) qui sert de pas spatial initial.
|
||||||
|
- `minPointsToContinue = 25` : la decimation s'arrete si un niveau tombe en dessous.
|
||||||
|
- `matchingCellsWarnThreshold = 90` : seuil d'affichage "a verifier" dans l'UI.
|
||||||
61
Dockerfile
Normal file
61
Dockerfile
Normal file
|
|
@ -0,0 +1,61 @@
|
||||||
|
# syntax=docker/dockerfile:1
|
||||||
|
|
||||||
|
# ---------- Etape 1 : build du backend NestJS ----------
|
||||||
|
FROM node:20-bookworm AS backend-build
|
||||||
|
WORKDIR /app
|
||||||
|
COPY package.json package-lock.json* ./
|
||||||
|
RUN npm install
|
||||||
|
COPY tsconfig.json tsconfig.build.json nest-cli.json eslint.config.mjs ./
|
||||||
|
COPY src ./src
|
||||||
|
# Portes de qualite : le build Docker (donc le deploiement Coolify) echoue si l'une de ces
|
||||||
|
# etapes echoue. Voir CLAUDE.md pour le detail des scripts.
|
||||||
|
RUN npm run lint
|
||||||
|
RUN npm run typecheck
|
||||||
|
RUN npm run test
|
||||||
|
RUN npm run build
|
||||||
|
RUN npm prune --omit=dev
|
||||||
|
|
||||||
|
# ---------- Etape 2 : image finale ----------
|
||||||
|
# Debian trixie fournit CloudCompare 2.13.2 directement en apt (paquet "cloudcompare"),
|
||||||
|
# ce qui evite une compilation depuis les sources (tres longue et fragile).
|
||||||
|
FROM debian:trixie-slim
|
||||||
|
|
||||||
|
ENV DEBIAN_FRONTEND=noninteractive \
|
||||||
|
NODE_ENV=production \
|
||||||
|
DATA_DIR=/data \
|
||||||
|
CLOUDCOMPARE_BIN=CloudCompare \
|
||||||
|
PYTHON_BIN=python3 \
|
||||||
|
DISPLAY=:99
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
cloudcompare \
|
||||||
|
xvfb \
|
||||||
|
x11-utils \
|
||||||
|
ca-certificates \
|
||||||
|
curl \
|
||||||
|
gnupg \
|
||||||
|
python3 \
|
||||||
|
python3-pip \
|
||||||
|
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
|
||||||
|
&& apt-get install -y --no-install-recommends nodejs \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY python/requirements.txt /app/python/requirements.txt
|
||||||
|
RUN pip3 install --break-system-packages --no-cache-dir -r /app/python/requirements.txt
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY --from=backend-build /app/node_modules ./node_modules
|
||||||
|
COPY --from=backend-build /app/dist ./dist
|
||||||
|
COPY package.json ./
|
||||||
|
COPY python ./python
|
||||||
|
COPY public ./public
|
||||||
|
COPY docker/entrypoint.sh /entrypoint.sh
|
||||||
|
RUN chmod +x /entrypoint.sh && mkdir -p /data
|
||||||
|
|
||||||
|
VOLUME ["/data"]
|
||||||
|
EXPOSE 3000
|
||||||
|
|
||||||
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
|
||||||
|
CMD curl -fsS "http://localhost:${PORT:-3000}/" || exit 1
|
||||||
|
|
||||||
|
ENTRYPOINT ["/entrypoint.sh"]
|
||||||
132
README.md
Normal file
132
README.md
Normal file
|
|
@ -0,0 +1,132 @@
|
||||||
|
# Volume 2.5D & robustesse a la decimation
|
||||||
|
|
||||||
|
Application locale (Docker) qui, a partir d'un nuage de points (LAS/LAZ/COPC.laz/BIN) :
|
||||||
|
|
||||||
|
1. calcule son volume en 2.5D via CloudCompare (plan de reference = altitude minimale du nuage) ;
|
||||||
|
2. le decime progressivement en methode **spatiale**, facteur **x2** sur **5 etapes** (le pas spatial double
|
||||||
|
a chaque etape, en partant d'un pas initial estime automatiquement = distance mediane au plus proche
|
||||||
|
voisin, calculee par KD-tree sur un echantillon) ;
|
||||||
|
3. recalcule le volume a chaque niveau avec la **meme** grille et le **meme** plan de reference, pour rendre
|
||||||
|
les 6 mesures comparables ;
|
||||||
|
4. assemble les 6 nuages (complet + 5 decimations) dans un seul fichier `.bin` CloudCompare, nomme et
|
||||||
|
consultable dans CloudCompare Desktop ;
|
||||||
|
5. produit un dossier de resultats : rapport CSV/JSON, cartes de hauteur par niveau, graphiques de
|
||||||
|
synthese (volume, nombre de points, robustesse) ;
|
||||||
|
6. expose tout ca dans une UI web simple avec historique des runs.
|
||||||
|
|
||||||
|
## Demarrage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
Puis ouvrir http://localhost:8080
|
||||||
|
|
||||||
|
Les donnees (uploads, resultats, base sqlite) sont persistees dans un volume Docker nomme `app-data`
|
||||||
|
(pas un bind mount vers `./data`, voir "Deploiement Coolify" ci-dessous pour la raison).
|
||||||
|
|
||||||
|
## Deploiement Coolify
|
||||||
|
|
||||||
|
Le repo est pret pour un deploiement Coolify via webhook (push sur `main` -> build + deploy auto) :
|
||||||
|
|
||||||
|
- Le `Dockerfile` fait tourner **lint + typecheck + tests unitaires** avant de compiler — un commit qui
|
||||||
|
casse l'un de ces trois echoue le build, donc n'est jamais deploye.
|
||||||
|
- `HEALTHCHECK` integre au Dockerfile (`curl` sur `/`) : Coolify l'utilise pour verifier que le
|
||||||
|
deploiement est reellement sain.
|
||||||
|
- Persistance : le volume `/data` doit etre monte comme un **volume nomme/gere par Coolify**, pas un
|
||||||
|
bind mount vers le checkout git (Coolify re-clone le repo a chaque deploiement — un bind mount
|
||||||
|
relatif au checkout perdrait toutes les donnees a chaque redeploy). Deux options :
|
||||||
|
- Resource **"Docker Compose"** dans Coolify pointant sur `docker-compose.yml` du repo : le volume
|
||||||
|
nomme `app-data` declare dedans est deja correct.
|
||||||
|
- Resource **"Application"** (Dockerfile) : configurer un volume persistant sur `/data` depuis
|
||||||
|
l'onglet "Storage" de Coolify, et le port expose sur `3000` (variable `PORT`, lue par l'appli).
|
||||||
|
- Variables d'environnement optionnelles a definir dans Coolify si besoin de les changer par rapport
|
||||||
|
aux defauts (voir tableau plus bas) : `MAX_UPLOAD_MB`, `STATS_SAMPLE_CAP`, `CC_TIMEOUT_MS`.
|
||||||
|
- Le webhook de deploiement automatique est deja configure cote Coolify sur ce repo.
|
||||||
|
|
||||||
|
## Utilisation
|
||||||
|
|
||||||
|
1. "Nouveau run" -> choisir un fichier `.las`, `.laz`, `.copc.laz` ou `.bin`.
|
||||||
|
2. Optionnel : override du plan de reference Z (par defaut : altitude minimale d'un echantillon du
|
||||||
|
nuage) et/ou du pas spatial initial (par defaut : distance mediane au plus proche voisin).
|
||||||
|
3. Le run tourne en arriere-plan (une file d'attente sequentielle traite les runs un par un). La page
|
||||||
|
de detail se rafraichit automatiquement (polling 2s).
|
||||||
|
4. Une fois termine : tableau des 6 niveaux (points, volume, surface, % de cellules de grille
|
||||||
|
"matching"), cartes de hauteur, graphiques de synthese, et telechargements (`.bin` fusionne,
|
||||||
|
`.csv`, dossier complet en `.zip`).
|
||||||
|
|
||||||
|
### Lecture des resultats
|
||||||
|
|
||||||
|
- **% de cellules "matching"** (rapport de volume 2.5D de CloudCompare) : indique la part de la grille
|
||||||
|
de calcul ou les deux surfaces (nuage / plan de reference) sont effectivement comparables. Quand la
|
||||||
|
decimation depasse la resolution de la grille de calcul, ce pourcentage chute — c'est le signal
|
||||||
|
principal de perte de robustesse. Un niveau est marque "a verifier" des que ce taux passe sous 90 %
|
||||||
|
(configurable via `MATCHING_CELLS_WARN_THRESHOLD` cote code, `config.ts`).
|
||||||
|
- La decimation s'arrete automatiquement si un niveau tombe sous 25 points exploitables ; les niveaux
|
||||||
|
suivants sont marques "skipped" avec la raison.
|
||||||
|
|
||||||
|
## Parametres (variables d'environnement, voir `docker-compose.yml`)
|
||||||
|
|
||||||
|
| Variable | Defaut | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| `PORT` | 3000 | port interne du serveur |
|
||||||
|
| `DATA_DIR` | `/data` | dossier de persistance (uploads, resultats, sqlite) |
|
||||||
|
| `MAX_UPLOAD_MB` | 2048 | taille max d'un nuage uploade |
|
||||||
|
| `STATS_SAMPLE_CAP` | 50000 | nb de points echantillonnes pour estimer le pas spatial initial |
|
||||||
|
| `CC_TIMEOUT_MS` | 600000 | timeout par appel CloudCompare CLI |
|
||||||
|
|
||||||
|
## Formats supportes
|
||||||
|
|
||||||
|
`.las`, `.laz`, `.copc.laz` (lu comme un LAZ standard — l'indexation octree COPC n'est pas utilisee,
|
||||||
|
seuls les points le sont), `.bin` (format natif CloudCompare).
|
||||||
|
|
||||||
|
**E57 non supporte pour l'instant** (exclu volontairement du perimetre initial).
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
- **Backend** : NestJS (TypeScript), file d'attente de jobs en memoire (un run a la fois), SQLite
|
||||||
|
(`better-sqlite3`) pour l'historique des runs.
|
||||||
|
- **Orchestration du pipeline CloudCompare** : chaque etape (estimation stats, conversion, decimation,
|
||||||
|
volume, export) est un appel independant au CLI `CloudCompare -SILENT ...` (le CLI fonctionne comme
|
||||||
|
une machine a etats sur UNE session de process, donc chaque etape = un process).
|
||||||
|
- **Scripts Python** (`python/`) : `stats_helper.py` (distance au plus proche voisin via scipy KD-tree),
|
||||||
|
`render_images.py` (cartes de hauteur + graphiques de synthese via matplotlib/numpy),
|
||||||
|
`las_to_xyz.py` (conversion LAS/LAZ/COPC -> ASCII, voir "Limitation connue" ci-dessous).
|
||||||
|
- **Frontend** : HTML/JS vanilla + Tailwind (CDN), servi en statique par NestJS. Pas de framework, pas
|
||||||
|
de build front separe.
|
||||||
|
- **Image Docker** : `debian:trixie-slim` + paquet apt `cloudcompare` (2.13.2, evite une compilation
|
||||||
|
depuis les sources) + `xvfb` (CloudCompare reste une appli Qt, meme en `-SILENT` elle a besoin d'un
|
||||||
|
display) + Node 20 + Python 3.
|
||||||
|
|
||||||
|
## Limitations connues
|
||||||
|
|
||||||
|
- **Le paquet apt `cloudcompare` (Debian) ne fournit pas le plugin LAS/LAZ** (uniquement le plugin
|
||||||
|
"Core I/O"). Consequence : les fichiers LAS/LAZ/COPC sont convertis en ASCII XYZ via `laspy` (Python)
|
||||||
|
*avant* d'etre transmis a CloudCompare, qui lit l'ASCII nativement sans plugin. Ca fonctionne bien
|
||||||
|
mais perd les attributs LAS annexes (intensite, classification, RGB...) — non utilises par ce calcul
|
||||||
|
de volume de toute facon.
|
||||||
|
- **Le meme paquet plante (assertion GDAL) sur l'export GeoTIFF natif** (`-RASTERIZE
|
||||||
|
-OUTPUT_RASTER_Z`). Les cartes de hauteur sont donc reconstruites nous-memes (export ASCII du nuage +
|
||||||
|
binning numpy/scipy + rendu matplotlib) plutot que de dependre du rasterizer CloudCompare.
|
||||||
|
- Le plan de reference Z et le pas de grille de volume sont **approximatifs** quand ils sont
|
||||||
|
auto-detectes (calcules sur un echantillon aleatoire de points, pas la totalite du nuage) — largement
|
||||||
|
suffisant pour comparer les 6 niveaux entre eux, mais a corriger via l'override si une valeur exacte
|
||||||
|
est necessaire.
|
||||||
|
- Un seul run traite a la fois (file d'attente sequentielle) : adapte a un usage local mono-utilisateur,
|
||||||
|
pas concu pour un usage concurrent multi-utilisateurs.
|
||||||
|
|
||||||
|
## Developpement local (sans Docker)
|
||||||
|
|
||||||
|
Necessite CloudCompare installe localement + Python 3 avec `pip install -r python/requirements.txt`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm install
|
||||||
|
npm run lint # eslint
|
||||||
|
npm run typecheck # tsc --noEmit
|
||||||
|
npm run test # jest (tests unitaires)
|
||||||
|
npm run build
|
||||||
|
CLOUDCOMPARE_BIN="/chemin/vers/CloudCompare" PYTHON_BIN=python3 DATA_DIR=./data node dist/main.js
|
||||||
|
```
|
||||||
|
|
||||||
|
Voir [CLAUDE.md](CLAUDE.md) pour le detail des comportements CloudCompare CLI verifies empiriquement
|
||||||
|
(pieges non documentes, limitations du paquet apt Debian, etc.) — a lire avant de toucher au pipeline.
|
||||||
19
docker-compose.yml
Normal file
19
docker-compose.yml
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
services:
|
||||||
|
volume-app:
|
||||||
|
build: .
|
||||||
|
ports:
|
||||||
|
- "8080:3000"
|
||||||
|
volumes:
|
||||||
|
# Volume nomme (pas un bind mount vers ./data) : Coolify re-clone le repo a chaque
|
||||||
|
# deploiement, un bind mount relatif au checkout perdrait les donnees a chaque redeploy.
|
||||||
|
- app-data:/data
|
||||||
|
environment:
|
||||||
|
- PORT=3000
|
||||||
|
- DATA_DIR=/data
|
||||||
|
- MAX_UPLOAD_MB=2048
|
||||||
|
- STATS_SAMPLE_CAP=50000
|
||||||
|
- CC_TIMEOUT_MS=600000
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
app-data:
|
||||||
23
docker/entrypoint.sh
Normal file
23
docker/entrypoint.sh
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
#!/bin/sh
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# CloudCompare reste une application Qt : meme en -SILENT elle a besoin d'un display.
|
||||||
|
# On demarre un Xvfb persistant pour tout le cycle de vie du conteneur plutot que
|
||||||
|
# d'en relancer un par appel CLI (plus rapide, evite les conflits de lockfile).
|
||||||
|
Xvfb "${DISPLAY:-:99}" -screen 0 1280x1024x24 -nolisten tcp &
|
||||||
|
XVFB_PID=$!
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
kill "$XVFB_PID" 2>/dev/null || true
|
||||||
|
}
|
||||||
|
trap cleanup EXIT INT TERM
|
||||||
|
|
||||||
|
# Attente que le display soit pret
|
||||||
|
for i in $(seq 1 20); do
|
||||||
|
if xdpyinfo -display "${DISPLAY:-:99}" >/dev/null 2>&1; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 0.25
|
||||||
|
done
|
||||||
|
|
||||||
|
exec node dist/main.js
|
||||||
15
eslint.config.mjs
Normal file
15
eslint.config.mjs
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
// @ts-check
|
||||||
|
import eslint from '@eslint/js';
|
||||||
|
import tseslint from 'typescript-eslint';
|
||||||
|
|
||||||
|
export default tseslint.config(
|
||||||
|
{ ignores: ['dist/**', 'node_modules/**', 'public/**'] },
|
||||||
|
eslint.configs.recommended,
|
||||||
|
...tseslint.configs.recommended,
|
||||||
|
{
|
||||||
|
rules: {
|
||||||
|
'@typescript-eslint/no-explicit-any': 'off',
|
||||||
|
'@typescript-eslint/no-unused-vars': ['warn', { argsIgnorePattern: '^_', varsIgnorePattern: '^_' }],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
9
nest-cli.json
Normal file
9
nest-cli.json
Normal file
|
|
@ -0,0 +1,9 @@
|
||||||
|
{
|
||||||
|
"$schema": "https://json.schemastore.org/nest-cli",
|
||||||
|
"collection": "@nestjs/schematics",
|
||||||
|
"sourceRoot": "src",
|
||||||
|
"compilerOptions": {
|
||||||
|
"deleteOutDir": true,
|
||||||
|
"assets": []
|
||||||
|
}
|
||||||
|
}
|
||||||
9826
package-lock.json
generated
Normal file
9826
package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load diff
52
package.json
Normal file
52
package.json
Normal file
|
|
@ -0,0 +1,52 @@
|
||||||
|
{
|
||||||
|
"name": "volume-decimation-app",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"description": "Calcul de volume 2.5D CloudCompare + test de robustesse par decimation spatiale progressive",
|
||||||
|
"private": true,
|
||||||
|
"scripts": {
|
||||||
|
"build": "nest build",
|
||||||
|
"start": "node dist/main.js",
|
||||||
|
"start:dev": "nest start --watch",
|
||||||
|
"format": "prettier --write \"src/**/*.ts\"",
|
||||||
|
"lint": "eslint \"src/**/*.ts\"",
|
||||||
|
"typecheck": "tsc --noEmit -p tsconfig.json",
|
||||||
|
"test": "jest"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@nestjs/common": "^10.4.15",
|
||||||
|
"@nestjs/core": "^10.4.15",
|
||||||
|
"@nestjs/platform-express": "^10.4.15",
|
||||||
|
"@nestjs/serve-static": "^4.0.2",
|
||||||
|
"archiver": "^7.0.1",
|
||||||
|
"better-sqlite3": "^11.8.1",
|
||||||
|
"multer": "^1.4.5-lts.1",
|
||||||
|
"reflect-metadata": "^0.2.2",
|
||||||
|
"rxjs": "^7.8.1",
|
||||||
|
"uuid": "^11.0.5"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@eslint/js": "^9.18.0",
|
||||||
|
"@nestjs/cli": "^10.4.9",
|
||||||
|
"@types/archiver": "^6.0.3",
|
||||||
|
"@types/better-sqlite3": "^7.6.13",
|
||||||
|
"@types/express": "^5.0.0",
|
||||||
|
"@types/jest": "^29.5.14",
|
||||||
|
"@types/multer": "^1.4.12",
|
||||||
|
"@types/node": "^22.10.7",
|
||||||
|
"@types/uuid": "^10.0.0",
|
||||||
|
"eslint": "^9.18.0",
|
||||||
|
"jest": "^29.7.0",
|
||||||
|
"ts-jest": "^29.2.5",
|
||||||
|
"typescript": "^5.7.3",
|
||||||
|
"typescript-eslint": "^8.20.0"
|
||||||
|
},
|
||||||
|
"jest": {
|
||||||
|
"moduleFileExtensions": ["js", "json", "ts"],
|
||||||
|
"rootDir": "src",
|
||||||
|
"testRegex": ".*\\.spec\\.ts$",
|
||||||
|
"transform": {
|
||||||
|
"^.+\\.(t|j)s$": "ts-jest"
|
||||||
|
},
|
||||||
|
"testEnvironment": "node"
|
||||||
|
}
|
||||||
|
}
|
||||||
238
public/app.js
Normal file
238
public/app.js
Normal file
|
|
@ -0,0 +1,238 @@
|
||||||
|
const API = '/api/runs';
|
||||||
|
const app = document.getElementById('app');
|
||||||
|
let pollTimer = null;
|
||||||
|
|
||||||
|
function stopPolling() {
|
||||||
|
if (pollTimer) { clearInterval(pollTimer); pollTimer = null; }
|
||||||
|
}
|
||||||
|
|
||||||
|
async function api(path, opts) {
|
||||||
|
const res = await fetch(path, opts);
|
||||||
|
if (!res.ok) {
|
||||||
|
let msg = res.statusText;
|
||||||
|
try { const j = await res.json(); msg = j.message || j.error || msg; } catch {}
|
||||||
|
throw new Error(msg);
|
||||||
|
}
|
||||||
|
return res.json();
|
||||||
|
}
|
||||||
|
|
||||||
|
function fmt(n, digits = 4) {
|
||||||
|
if (n === null || n === undefined) return '-';
|
||||||
|
if (typeof n !== 'number') return String(n);
|
||||||
|
return n.toLocaleString('fr-FR', { maximumFractionDigits: digits });
|
||||||
|
}
|
||||||
|
|
||||||
|
function statusBadge(status) {
|
||||||
|
const map = {
|
||||||
|
pending: 'bg-slate-700 text-slate-200',
|
||||||
|
running: 'bg-blue-600/30 text-blue-300',
|
||||||
|
done: 'bg-emerald-600/30 text-emerald-300',
|
||||||
|
error: 'bg-red-600/30 text-red-300',
|
||||||
|
};
|
||||||
|
return `<span class="badge ${map[status] || 'bg-slate-700'}">${status}</span>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function router() {
|
||||||
|
const hash = location.hash || '#/';
|
||||||
|
stopPolling();
|
||||||
|
if (hash === '#/' || hash === '') return renderList();
|
||||||
|
if (hash === '#/new') return renderNew();
|
||||||
|
const m = hash.match(/^#\/runs\/([^/]+)$/);
|
||||||
|
if (m) return renderDetail(m[1]);
|
||||||
|
app.innerHTML = '<p class="text-slate-400">Page introuvable.</p>';
|
||||||
|
}
|
||||||
|
window.addEventListener('hashchange', router);
|
||||||
|
window.addEventListener('DOMContentLoaded', router);
|
||||||
|
|
||||||
|
// ---------------- Liste des runs ----------------
|
||||||
|
async function renderList() {
|
||||||
|
app.innerHTML = '<p class="text-slate-400">Chargement...</p>';
|
||||||
|
const runs = await api(API);
|
||||||
|
if (runs.length === 0) {
|
||||||
|
app.innerHTML = `
|
||||||
|
<div class="text-center py-20">
|
||||||
|
<p class="text-slate-400 mb-4">Aucun run pour le moment.</p>
|
||||||
|
<a href="#/new" class="inline-block bg-blue-600 hover:bg-blue-500 px-4 py-2 rounded-lg font-medium">Lancer un nouveau run</a>
|
||||||
|
</div>`;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const rows = runs.map((r) => `
|
||||||
|
<tr class="border-b border-slate-800 hover:bg-slate-900/50 cursor-pointer" onclick="location.hash='#/runs/${r.id}'">
|
||||||
|
<td class="py-2 px-3 font-mono text-xs text-slate-400">${r.id.slice(0, 8)}</td>
|
||||||
|
<td class="py-2 px-3">${escapeHtml(r.originalFilename)}</td>
|
||||||
|
<td class="py-2 px-3">${statusBadge(r.status)}</td>
|
||||||
|
<td class="py-2 px-3 text-sm text-slate-400">${new Date(r.createdAt).toLocaleString('fr-FR')}</td>
|
||||||
|
<td class="py-2 px-3 text-sm">${r.summary ? `${r.summary.levelsOk}/${r.summary.levelsTotal} niveaux` : '-'}</td>
|
||||||
|
<td class="py-2 px-3 text-sm">${r.summary && r.summary.volumeDeltaPct !== null ? fmt(r.summary.volumeDeltaPct, 2) + ' %' : '-'}</td>
|
||||||
|
</tr>`).join('');
|
||||||
|
|
||||||
|
app.innerHTML = `
|
||||||
|
<div class="flex items-center justify-between mb-6">
|
||||||
|
<h1 class="text-xl font-semibold">Runs</h1>
|
||||||
|
<a href="#/new" class="bg-blue-600 hover:bg-blue-500 px-4 py-2 rounded-lg font-medium text-sm">+ Nouveau run</a>
|
||||||
|
</div>
|
||||||
|
<div class="overflow-x-auto border border-slate-800 rounded-lg">
|
||||||
|
<table class="w-full text-left">
|
||||||
|
<thead class="bg-slate-900 text-slate-400 text-xs uppercase">
|
||||||
|
<tr>
|
||||||
|
<th class="py-2 px-3">ID</th><th class="py-2 px-3">Fichier</th><th class="py-2 px-3">Statut</th>
|
||||||
|
<th class="py-2 px-3">Cree le</th><th class="py-2 px-3">Niveaux OK</th><th class="py-2 px-3">Ecart volume (L0 -> dernier)</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>${rows}</tbody>
|
||||||
|
</table>
|
||||||
|
</div>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------- Nouveau run ----------------
|
||||||
|
function renderNew() {
|
||||||
|
app.innerHTML = `
|
||||||
|
<h1 class="text-xl font-semibold mb-6">Nouveau run</h1>
|
||||||
|
<form id="upload-form" class="space-y-5 max-w-xl">
|
||||||
|
<div>
|
||||||
|
<label class="block text-sm font-medium mb-1">Nuage de points (.las, .laz, .copc.laz, .bin)</label>
|
||||||
|
<input required type="file" name="file" accept=".las,.laz,.bin"
|
||||||
|
class="block w-full text-sm text-slate-300 border border-slate-700 rounded-lg cursor-pointer bg-slate-900 focus:outline-none p-2" />
|
||||||
|
</div>
|
||||||
|
<details class="border border-slate-800 rounded-lg p-3">
|
||||||
|
<summary class="cursor-pointer text-sm text-slate-400">Options avancees (optionnel)</summary>
|
||||||
|
<div class="mt-3 space-y-3">
|
||||||
|
<div>
|
||||||
|
<label class="block text-sm mb-1">Plan de reference Z (override)</label>
|
||||||
|
<input type="number" step="any" name="zrefOverride" placeholder="auto = Zmin de l'echantillon"
|
||||||
|
class="w-full bg-slate-900 border border-slate-700 rounded-lg p-2 text-sm" />
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label class="block text-sm mb-1">Pas spatial initial en metres (override)</label>
|
||||||
|
<input type="number" step="any" name="initialStepOverride" placeholder="auto = distance mediane au plus proche voisin"
|
||||||
|
class="w-full bg-slate-900 border border-slate-700 rounded-lg p-2 text-sm" />
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</details>
|
||||||
|
<button type="submit" class="bg-blue-600 hover:bg-blue-500 px-4 py-2 rounded-lg font-medium">Lancer le run</button>
|
||||||
|
<p id="upload-error" class="text-red-400 text-sm hidden"></p>
|
||||||
|
</form>`;
|
||||||
|
|
||||||
|
document.getElementById('upload-form').addEventListener('submit', async (e) => {
|
||||||
|
e.preventDefault();
|
||||||
|
const form = e.target;
|
||||||
|
const fd = new FormData(form);
|
||||||
|
const errEl = document.getElementById('upload-error');
|
||||||
|
errEl.classList.add('hidden');
|
||||||
|
const btn = form.querySelector('button[type=submit]');
|
||||||
|
btn.disabled = true;
|
||||||
|
btn.textContent = 'Envoi...';
|
||||||
|
try {
|
||||||
|
const res = await fetch(API, { method: 'POST', body: fd });
|
||||||
|
const json = await res.json();
|
||||||
|
if (!res.ok) throw new Error(json.message || 'Echec de l\'envoi');
|
||||||
|
location.hash = `#/runs/${json.id}`;
|
||||||
|
} catch (err) {
|
||||||
|
errEl.textContent = err.message;
|
||||||
|
errEl.classList.remove('hidden');
|
||||||
|
btn.disabled = false;
|
||||||
|
btn.textContent = 'Lancer le run';
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------- Detail d'un run ----------------
|
||||||
|
async function renderDetail(id) {
|
||||||
|
app.innerHTML = '<p class="text-slate-400">Chargement...</p>';
|
||||||
|
await loadDetail(id);
|
||||||
|
pollTimer = setInterval(async () => {
|
||||||
|
const run = await api(`${API}/${id}`);
|
||||||
|
if (run.status === 'pending' || run.status === 'running') {
|
||||||
|
paintDetail(run);
|
||||||
|
} else {
|
||||||
|
stopPolling();
|
||||||
|
paintDetail(run);
|
||||||
|
}
|
||||||
|
}, 2000);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadDetail(id) {
|
||||||
|
const run = await api(`${API}/${id}`);
|
||||||
|
paintDetail(run);
|
||||||
|
}
|
||||||
|
|
||||||
|
function levelRow(l, id) {
|
||||||
|
const warnCls = l.status === 'error' ? 'bg-red-950/40' : l.warn ? 'bg-amber-950/30' : '';
|
||||||
|
const statusTxt = l.status === 'ok' ? '' : `<div class="text-xs text-slate-400 mt-0.5">${escapeHtml(l.message || '')}</div>`;
|
||||||
|
return `
|
||||||
|
<tr class="border-b border-slate-800 ${warnCls}">
|
||||||
|
<td class="py-2 px-3 font-medium">${l.level}</td>
|
||||||
|
<td class="py-2 px-3">${l.spatialStep !== null ? fmt(l.spatialStep, 5) + ' m' : 'complet'}</td>
|
||||||
|
<td class="py-2 px-3">${l.status === 'ok' ? l.pointCount.toLocaleString('fr-FR') : '-'}</td>
|
||||||
|
<td class="py-2 px-3">${l.status === 'ok' ? fmt(l.pointRatio * 100, 2) + ' %' : '-'}</td>
|
||||||
|
<td class="py-2 px-3">${l.status === 'ok' ? fmt(l.volume, 4) + ' m3' : '-'}</td>
|
||||||
|
<td class="py-2 px-3">${l.status === 'ok' ? fmt(l.surface, 3) + ' m2' : '-'}</td>
|
||||||
|
<td class="py-2 px-3">${l.status === 'ok' ? fmt(l.matchingCellsPct, 1) + ' %' : '-'}</td>
|
||||||
|
<td class="py-2 px-3">${statusBadge(l.status)} ${l.warn && l.status === 'ok' ? '<span class="badge bg-amber-600/30 text-amber-300 ml-1">a verifier</span>' : ''}${statusTxt}</td>
|
||||||
|
</tr>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function paintDetail(run) {
|
||||||
|
const r = run.report;
|
||||||
|
const levelsTable = r ? `
|
||||||
|
<div class="overflow-x-auto border border-slate-800 rounded-lg mt-4">
|
||||||
|
<table class="w-full text-left text-sm">
|
||||||
|
<thead class="bg-slate-900 text-slate-400 text-xs uppercase">
|
||||||
|
<tr>
|
||||||
|
<th class="py-2 px-3">Niveau</th><th class="py-2 px-3">Pas spatial</th><th class="py-2 px-3">Points</th>
|
||||||
|
<th class="py-2 px-3">% points vs L0</th><th class="py-2 px-3">Volume</th><th class="py-2 px-3">Surface</th>
|
||||||
|
<th class="py-2 px-3">Matching cells</th><th class="py-2 px-3">Statut</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>${r.levels.map((l) => levelRow(l, run.id)).join('')}</tbody>
|
||||||
|
</table>
|
||||||
|
</div>` : '';
|
||||||
|
|
||||||
|
const params = r ? `
|
||||||
|
<dl class="grid grid-cols-2 sm:grid-cols-4 gap-3 text-sm mt-4">
|
||||||
|
<div class="bg-slate-900 border border-slate-800 rounded-lg p-3"><dt class="text-slate-400 text-xs">Zref</dt><dd>${fmt(r.params.zref, 4)} (${r.params.zrefSource})</dd></div>
|
||||||
|
<div class="bg-slate-900 border border-slate-800 rounded-lg p-3"><dt class="text-slate-400 text-xs">Pas initial</dt><dd>${fmt(r.params.initialStep, 5)} m (${r.params.initialStepSource})</dd></div>
|
||||||
|
<div class="bg-slate-900 border border-slate-800 rounded-lg p-3"><dt class="text-slate-400 text-xs">Grille volume</dt><dd>${fmt(r.params.gridStep, 5)} m</dd></div>
|
||||||
|
<div class="bg-slate-900 border border-slate-800 rounded-lg p-3"><dt class="text-slate-400 text-xs">Facteur / etapes</dt><dd>x${r.params.factor} sur ${r.params.steps}</dd></div>
|
||||||
|
</dl>` : '';
|
||||||
|
|
||||||
|
const heightmaps = r ? r.levels.filter((l) => l.imageFile).map((l) => `
|
||||||
|
<div class="bg-slate-900 border border-slate-800 rounded-lg p-2">
|
||||||
|
<img src="${API}/${run.id}/images/${l.imageFile.split('/').pop()}" class="w-full rounded" loading="lazy" />
|
||||||
|
<p class="text-xs text-slate-400 text-center mt-1">Niveau ${l.level}</p>
|
||||||
|
</div>`).join('') : '';
|
||||||
|
|
||||||
|
const summaryCharts = run.status === 'done' ? ['volume_vs_level.png', 'points_vs_level.png', 'robustness_vs_level.png'].map((f) => `
|
||||||
|
<div class="bg-slate-900 border border-slate-800 rounded-lg p-2">
|
||||||
|
<img src="${API}/${run.id}/images/${f}" class="w-full rounded" loading="lazy" onerror="this.parentElement.style.display='none'" />
|
||||||
|
</div>`).join('') : '';
|
||||||
|
|
||||||
|
const downloads = run.status === 'done' ? `
|
||||||
|
<div class="flex flex-wrap gap-2 mt-4">
|
||||||
|
<a href="${API}/${run.id}/download/bin" class="bg-slate-800 hover:bg-slate-700 px-3 py-2 rounded-lg text-sm">Nuage assemble (.bin)</a>
|
||||||
|
<a href="${API}/${run.id}/download/csv" class="bg-slate-800 hover:bg-slate-700 px-3 py-2 rounded-lg text-sm">Rapport (.csv)</a>
|
||||||
|
<a href="${API}/${run.id}/download/zip" class="bg-slate-800 hover:bg-slate-700 px-3 py-2 rounded-lg text-sm">Dossier complet (.zip)</a>
|
||||||
|
</div>` : '';
|
||||||
|
|
||||||
|
const errorBlock = run.error ? `<div class="bg-red-950/40 border border-red-900 rounded-lg p-3 text-sm text-red-300 mt-4">${escapeHtml(run.error)}</div>` : '';
|
||||||
|
|
||||||
|
app.innerHTML = `
|
||||||
|
<div class="flex items-center justify-between mb-2">
|
||||||
|
<h1 class="text-xl font-semibold">${escapeHtml(run.originalFilename)}</h1>
|
||||||
|
${statusBadge(run.status)}
|
||||||
|
</div>
|
||||||
|
<p class="text-slate-400 text-sm mb-4">Cree le ${new Date(run.createdAt).toLocaleString('fr-FR')}${run.finishedAt ? ' - termine le ' + new Date(run.finishedAt).toLocaleString('fr-FR') : ''}</p>
|
||||||
|
${errorBlock}
|
||||||
|
${params}
|
||||||
|
${levelsTable}
|
||||||
|
${downloads}
|
||||||
|
${heightmaps ? `<h2 class="text-lg font-semibold mt-8 mb-3">Cartes de hauteur par niveau</h2><div class="grid grid-cols-2 md:grid-cols-3 gap-3">${heightmaps}</div>` : ''}
|
||||||
|
${summaryCharts ? `<h2 class="text-lg font-semibold mt-8 mb-3">Graphiques de synthese</h2><div class="grid grid-cols-1 md:grid-cols-3 gap-3">${summaryCharts}</div>` : ''}
|
||||||
|
<h2 class="text-lg font-semibold mt-8 mb-2">Journal</h2>
|
||||||
|
<pre class="bg-black/60 border border-slate-800 rounded-lg p-3 text-xs text-slate-300 overflow-x-auto max-h-64 overflow-y-auto">${escapeHtml(run.log || '')}</pre>
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function escapeHtml(s) {
|
||||||
|
return String(s).replace(/[&<>"']/g, (c) => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' }[c]));
|
||||||
|
}
|
||||||
28
public/index.html
Normal file
28
public/index.html
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
<!doctype html>
|
||||||
|
<html lang="fr">
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||||
|
<title>Volume 2.5D & robustesse a la decimation</title>
|
||||||
|
<script src="https://cdn.tailwindcss.com"></script>
|
||||||
|
<style>
|
||||||
|
body { font-family: ui-sans-serif, system-ui, -apple-system, sans-serif; }
|
||||||
|
.badge { display: inline-flex; align-items: center; padding: 0.125rem 0.5rem; border-radius: 9999px; font-size: 0.75rem; font-weight: 600; }
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body class="bg-slate-950 text-slate-100 min-h-screen">
|
||||||
|
<header class="border-b border-slate-800 bg-slate-900/60 sticky top-0 backdrop-blur z-10">
|
||||||
|
<div class="max-w-6xl mx-auto px-4 py-3 flex items-center justify-between">
|
||||||
|
<a href="#/" class="text-lg font-semibold tracking-tight">Volume 2.5D & robustesse decimation</a>
|
||||||
|
<nav class="text-sm text-slate-400 space-x-4">
|
||||||
|
<a href="#/" class="hover:text-slate-100">Runs</a>
|
||||||
|
<a href="#/new" class="hover:text-slate-100">Nouveau run</a>
|
||||||
|
</nav>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<main id="app" class="max-w-6xl mx-auto px-4 py-8"></main>
|
||||||
|
|
||||||
|
<script src="app.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
47
python/las_to_xyz.py
Normal file
47
python/las_to_xyz.py
Normal file
|
|
@ -0,0 +1,47 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Convertit un fichier LAS/LAZ/COPC (via laspy) en nuage ASCII XYZ brut, lisible
|
||||||
|
nativement par CloudCompare sans plugin LAS (le paquet apt Debian de CloudCompare
|
||||||
|
ne fournit pas le plugin LAS/LAZ - seulement le plugin "Core I/O").
|
||||||
|
|
||||||
|
Un .copc.laz est simplement lu comme un LAZ standard (les points sont
|
||||||
|
identiques, seule l'indexation octree COPC, inutile ici, est ignoree).
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python3 las_to_xyz.py --input nuage.laz --output nuage.xyz
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import laspy
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--input", required=True)
|
||||||
|
parser.add_argument("--output", required=True)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
try:
|
||||||
|
total = 0
|
||||||
|
with laspy.open(args.input) as reader, open(args.output, "w", encoding="ascii") as out:
|
||||||
|
for chunk in reader.chunk_iterator(2_000_000):
|
||||||
|
xyz = np.column_stack([np.asarray(chunk.x), np.asarray(chunk.y), np.asarray(chunk.z)])
|
||||||
|
np.savetxt(out, xyz, fmt="%.6f")
|
||||||
|
total += xyz.shape[0]
|
||||||
|
except Exception as exc:
|
||||||
|
print(json.dumps({"error": f"conversion LAS/LAZ impossible: {exc}"}))
|
||||||
|
return 1
|
||||||
|
|
||||||
|
if total == 0:
|
||||||
|
print(json.dumps({"error": "le fichier LAS/LAZ ne contient aucun point"}))
|
||||||
|
return 1
|
||||||
|
|
||||||
|
print(json.dumps({"ok": True, "count": total}))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
155
python/render_images.py
Normal file
155
python/render_images.py
Normal file
|
|
@ -0,0 +1,155 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Generation des images du dossier de resultats:
|
||||||
|
- "heightmap" : construit une carte de hauteur (grille 2.5D) coloree en PNG directement a partir
|
||||||
|
d'un nuage ASCII XYZ (binning numpy). Le paquet apt CloudCompare (Debian) plante
|
||||||
|
sur l'export GeoTIFF (assertion GDAL manquante) - on reconstruit donc la grille
|
||||||
|
nous-memes plutot que de dependre de -RASTERIZE -OUTPUT_RASTER_Z.
|
||||||
|
- "summary" : genere les graphiques de synthese (volume/points/robustesse vs niveau de decimation)
|
||||||
|
a partir du report.json produit par le pipeline.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python3 render_images.py heightmap --input cloud.xyz --gridstep 0.05 --output heightmap.png --title "Niveau 0"
|
||||||
|
python3 render_images.py summary --report report.json --outdir images/
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import matplotlib
|
||||||
|
|
||||||
|
matplotlib.use("Agg")
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
import numpy as np
|
||||||
|
from scipy.stats import binned_statistic_2d
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_heightmap(args: argparse.Namespace) -> int:
|
||||||
|
try:
|
||||||
|
pts = np.loadtxt(args.input, dtype=np.float64, usecols=(0, 1, 2))
|
||||||
|
except Exception as exc:
|
||||||
|
print(json.dumps({"error": f"lecture xyz impossible: {exc}"}))
|
||||||
|
return 1
|
||||||
|
|
||||||
|
if pts.ndim == 1:
|
||||||
|
pts = pts.reshape(1, -1)
|
||||||
|
|
||||||
|
x, y, z = pts[:, 0], pts[:, 1], pts[:, 2]
|
||||||
|
xmin, xmax = float(np.min(x)), float(np.max(x))
|
||||||
|
ymin, ymax = float(np.min(y)), float(np.max(y))
|
||||||
|
nx = max(2, int(np.ceil((xmax - xmin) / args.gridstep)) + 1) if xmax > xmin else 2
|
||||||
|
ny = max(2, int(np.ceil((ymax - ymin) / args.gridstep)) + 1) if ymax > ymin else 2
|
||||||
|
# Grille bornee pour eviter une image demesuree si le pas est mal renseigne
|
||||||
|
nx, ny = min(nx, 2000), min(ny, 2000)
|
||||||
|
|
||||||
|
arr = None
|
||||||
|
if pts.shape[0] >= 1 and xmax > xmin and ymax > ymin:
|
||||||
|
stat, _, _, _ = binned_statistic_2d(x, y, z, statistic="mean", bins=[nx, ny])
|
||||||
|
arr = stat.T
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(6, 5.2), dpi=130)
|
||||||
|
if arr is None or not np.isfinite(arr).any():
|
||||||
|
ax.text(0.5, 0.5, "Aucune donnee exploitable", ha="center", va="center")
|
||||||
|
else:
|
||||||
|
im = ax.imshow(arr, cmap="terrain", origin="lower", extent=[xmin, xmax, ymin, ymax], aspect="auto")
|
||||||
|
cbar = fig.colorbar(im, ax=ax, shrink=0.85)
|
||||||
|
cbar.set_label("Altitude (m, moyenne par cellule)")
|
||||||
|
ax.set_title(args.title or Path(args.input).stem)
|
||||||
|
ax.set_xlabel("X (m)")
|
||||||
|
ax.set_ylabel("Y (m)")
|
||||||
|
fig.tight_layout()
|
||||||
|
fig.savefig(args.output)
|
||||||
|
plt.close(fig)
|
||||||
|
print(json.dumps({"ok": True, "output": args.output}))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_summary(args: argparse.Namespace) -> int:
|
||||||
|
with open(args.report, "r", encoding="utf-8") as f:
|
||||||
|
report = json.load(f)
|
||||||
|
|
||||||
|
levels = [lvl for lvl in report.get("levels", []) if lvl.get("status") == "ok"]
|
||||||
|
if not levels:
|
||||||
|
print(json.dumps({"error": "aucun niveau exploitable dans le rapport"}))
|
||||||
|
return 1
|
||||||
|
|
||||||
|
outdir = Path(args.outdir)
|
||||||
|
outdir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
xs = [lvl["level"] for lvl in levels]
|
||||||
|
volumes = [lvl["volume"] for lvl in levels]
|
||||||
|
points = [lvl["pointCount"] for lvl in levels]
|
||||||
|
matching = [lvl["matchingCellsPct"] for lvl in levels]
|
||||||
|
v0 = volumes[0] if volumes else None
|
||||||
|
|
||||||
|
# --- Volume vs niveau ---
|
||||||
|
fig, ax = plt.subplots(figsize=(6, 4), dpi=130)
|
||||||
|
ax.plot(xs, volumes, marker="o", color="#2563eb")
|
||||||
|
ax.set_xlabel("Niveau de decimation")
|
||||||
|
ax.set_ylabel("Volume (m3)")
|
||||||
|
ax.set_title("Volume calcule vs niveau de decimation")
|
||||||
|
ax.set_xticks(xs)
|
||||||
|
ax.grid(alpha=0.3)
|
||||||
|
fig.tight_layout()
|
||||||
|
fig.savefig(outdir / "volume_vs_level.png")
|
||||||
|
plt.close(fig)
|
||||||
|
|
||||||
|
# --- Nombre de points (log) vs niveau ---
|
||||||
|
fig, ax = plt.subplots(figsize=(6, 4), dpi=130)
|
||||||
|
ax.semilogy(xs, points, marker="o", color="#16a34a")
|
||||||
|
ax.set_xlabel("Niveau de decimation")
|
||||||
|
ax.set_ylabel("Nombre de points (log)")
|
||||||
|
ax.set_title("Nombre de points vs niveau de decimation")
|
||||||
|
ax.set_xticks(xs)
|
||||||
|
ax.grid(alpha=0.3, which="both")
|
||||||
|
fig.tight_layout()
|
||||||
|
fig.savefig(outdir / "points_vs_level.png")
|
||||||
|
plt.close(fig)
|
||||||
|
|
||||||
|
# --- Ecart relatif de volume + robustesse (matching cells %) vs niveau ---
|
||||||
|
fig, ax1 = plt.subplots(figsize=(6, 4), dpi=130)
|
||||||
|
if v0:
|
||||||
|
delta_pct = [100.0 * (v - v0) / v0 for v in volumes]
|
||||||
|
ax1.plot(xs, delta_pct, marker="o", color="#dc2626", label="Ecart volume vs niveau 0 (%)")
|
||||||
|
ax1.set_xlabel("Niveau de decimation")
|
||||||
|
ax1.set_ylabel("Ecart volume (%)", color="#dc2626")
|
||||||
|
ax1.set_xticks(xs)
|
||||||
|
ax1.grid(alpha=0.3)
|
||||||
|
|
||||||
|
ax2 = ax1.twinx()
|
||||||
|
ax2.plot(xs, matching, marker="s", linestyle="--", color="#7c3aed", label="Matching cells (%)")
|
||||||
|
ax2.set_ylabel("Matching cells (%)", color="#7c3aed")
|
||||||
|
ax2.set_ylim(0, 105)
|
||||||
|
|
||||||
|
fig.suptitle("Robustesse : ecart de volume et couverture de grille")
|
||||||
|
fig.tight_layout()
|
||||||
|
fig.savefig(outdir / "robustness_vs_level.png")
|
||||||
|
plt.close(fig)
|
||||||
|
|
||||||
|
print(json.dumps({"ok": True, "outdir": str(outdir)}))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
sub = parser.add_subparsers(dest="command", required=True)
|
||||||
|
|
||||||
|
p_heightmap = sub.add_parser("heightmap")
|
||||||
|
p_heightmap.add_argument("--input", required=True)
|
||||||
|
p_heightmap.add_argument("--gridstep", required=True, type=float)
|
||||||
|
p_heightmap.add_argument("--output", required=True)
|
||||||
|
p_heightmap.add_argument("--title", default=None)
|
||||||
|
p_heightmap.set_defaults(func=cmd_heightmap)
|
||||||
|
|
||||||
|
p_summary = sub.add_parser("summary")
|
||||||
|
p_summary.add_argument("--report", required=True)
|
||||||
|
p_summary.add_argument("--outdir", required=True)
|
||||||
|
p_summary.set_defaults(func=cmd_summary)
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
return args.func(args)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
5
python/requirements.txt
Normal file
5
python/requirements.txt
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
numpy>=1.26,<2.3
|
||||||
|
laspy[lazrs]>=2.5,<2.6
|
||||||
|
scipy>=1.11,<1.15
|
||||||
|
Pillow>=10.0,<11.0
|
||||||
|
matplotlib>=3.8,<3.10
|
||||||
72
python/stats_helper.py
Normal file
72
python/stats_helper.py
Normal file
|
|
@ -0,0 +1,72 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Calcule des statistiques (distance moyenne/mediane au plus proche voisin, bbox)
|
||||||
|
a partir d'un echantillon de points XYZ (fichier ASCII exporte par CloudCompare).
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python3 stats_helper.py --input sample.xyz
|
||||||
|
|
||||||
|
Sortie: un objet JSON unique sur stdout.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
from scipy.spatial import cKDTree
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--input", required=True, help="Fichier ASCII XYZ (espace separe)")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
try:
|
||||||
|
pts = np.loadtxt(args.input, dtype=np.float64, usecols=(0, 1, 2))
|
||||||
|
except Exception as exc: # fichier vide, format inattendu, etc.
|
||||||
|
print(json.dumps({"error": f"lecture impossible: {exc}"}))
|
||||||
|
return 1
|
||||||
|
|
||||||
|
if pts.ndim == 1:
|
||||||
|
# une seule ligne -> un seul point
|
||||||
|
pts = pts.reshape(1, -1)
|
||||||
|
|
||||||
|
count = int(pts.shape[0])
|
||||||
|
if count < 2:
|
||||||
|
print(json.dumps({"error": f"pas assez de points dans l'echantillon ({count})"}))
|
||||||
|
return 1
|
||||||
|
|
||||||
|
bbox = {
|
||||||
|
"xmin": float(np.min(pts[:, 0])),
|
||||||
|
"xmax": float(np.max(pts[:, 0])),
|
||||||
|
"ymin": float(np.min(pts[:, 1])),
|
||||||
|
"ymax": float(np.max(pts[:, 1])),
|
||||||
|
"zmin": float(np.min(pts[:, 2])),
|
||||||
|
"zmax": float(np.max(pts[:, 2])),
|
||||||
|
}
|
||||||
|
|
||||||
|
tree = cKDTree(pts)
|
||||||
|
# k=2 : le plus proche voisin de chaque point est le point lui-meme (distance 0),
|
||||||
|
# le 2e plus proche est le vrai plus proche voisin.
|
||||||
|
dists, _ = tree.query(pts, k=2, workers=-1)
|
||||||
|
nn_dist = dists[:, 1]
|
||||||
|
nn_dist = nn_dist[np.isfinite(nn_dist) & (nn_dist > 0)]
|
||||||
|
|
||||||
|
if nn_dist.size == 0:
|
||||||
|
print(json.dumps({"error": "distances au plus proche voisin toutes nulles/invalides (points dupliques ?)"}))
|
||||||
|
return 1
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"count": count,
|
||||||
|
"mean_nn": float(np.mean(nn_dist)),
|
||||||
|
"median_nn": float(np.median(nn_dist)),
|
||||||
|
"std_nn": float(np.std(nn_dist)),
|
||||||
|
"bbox": bbox,
|
||||||
|
"approximate": True,
|
||||||
|
}
|
||||||
|
print(json.dumps(result))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
15
src/app.module.ts
Normal file
15
src/app.module.ts
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
import { Module } from '@nestjs/common';
|
||||||
|
import { ServeStaticModule } from '@nestjs/serve-static';
|
||||||
|
import * as path from 'path';
|
||||||
|
import { RunsModule } from './runs/runs.module';
|
||||||
|
|
||||||
|
@Module({
|
||||||
|
imports: [
|
||||||
|
ServeStaticModule.forRoot({
|
||||||
|
rootPath: path.resolve(__dirname, '..', 'public'),
|
||||||
|
exclude: ['/api*'],
|
||||||
|
}),
|
||||||
|
RunsModule,
|
||||||
|
],
|
||||||
|
})
|
||||||
|
export class AppModule {}
|
||||||
122
src/cloudcompare/cc-runner.service.spec.ts
Normal file
122
src/cloudcompare/cc-runner.service.spec.ts
Normal file
|
|
@ -0,0 +1,122 @@
|
||||||
|
import * as fs from 'fs/promises';
|
||||||
|
import * as os from 'os';
|
||||||
|
import * as path from 'path';
|
||||||
|
import { CcRunnerService } from './cc-runner.service';
|
||||||
|
|
||||||
|
describe('CcRunnerService', () => {
|
||||||
|
const service = new CcRunnerService();
|
||||||
|
|
||||||
|
describe('parseLoadedPointCount', () => {
|
||||||
|
it('extrait le nombre de points du premier "Found one cloud with N points"', () => {
|
||||||
|
const log = `
|
||||||
|
[12:56:04] [LOAD]
|
||||||
|
[12:56:04] Opening file: 'test_cloud.las'
|
||||||
|
[12:56:04] Found one cloud with 250000 points
|
||||||
|
[12:56:04] [LOAD] finished in 0.02 s.
|
||||||
|
`;
|
||||||
|
expect(service.parseLoadedPointCount(log)).toBe(250000);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('renvoie null si le nuage n\'a pas pu etre charge', () => {
|
||||||
|
const log = `[13:18:20] [Load] Can't guess file format: unhandled file extension 'las'`;
|
||||||
|
expect(service.parseLoadedPointCount(log)).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('parseSubsampleResult', () => {
|
||||||
|
it('extrait le nombre de points apres -SS SPATIAL/RANDOM', () => {
|
||||||
|
const log = `
|
||||||
|
[12:56:30] [SUBSAMPLE]
|
||||||
|
[12:56:30] \tResult: 25451 points
|
||||||
|
[12:56:30] [SUBSAMPLE] finished in 0.60 s.
|
||||||
|
`;
|
||||||
|
expect(service.parseSubsampleResult(log)).toBe(25451);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('renvoie null si aucune section [SUBSAMPLE] n\'est presente', () => {
|
||||||
|
expect(service.parseSubsampleResult('rien a voir ici')).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('parseVolumeReportFile', () => {
|
||||||
|
// Format verifie empiriquement avec CloudCompare 2.13.2 (voir CLAUDE.md) :
|
||||||
|
// -VOLUME ne genere ce fichier que si AUTO_SAVE est ON.
|
||||||
|
const reportContent = [
|
||||||
|
'1\tVolume: 7.834176',
|
||||||
|
'2\tSurface: 100.774998',
|
||||||
|
'3\t----------------------',
|
||||||
|
'4\tAdded volume: (+)7.975798',
|
||||||
|
'5\tRemoved volume: (-)0.141622',
|
||||||
|
'6\t----------------------',
|
||||||
|
'7\tMatching cells: 99.8%',
|
||||||
|
'8\tNon-matching cells:',
|
||||||
|
'9\t ground = 0.2%',
|
||||||
|
'10\t ceil = 0.0%',
|
||||||
|
'11\tAverage neighbors per cell: 8.0 / 8.0',
|
||||||
|
'12\t',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
let tmpFile: string;
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
tmpFile = path.join(os.tmpdir(), `VolumeCalculationReport_test_${Date.now()}.txt`);
|
||||||
|
await fs.writeFile(tmpFile, reportContent, 'utf-8');
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
await fs.unlink(tmpFile).catch(() => undefined);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('parse toutes les valeurs du rapport CloudCompare', async () => {
|
||||||
|
const result = await service.parseVolumeReportFile(tmpFile);
|
||||||
|
expect(result).toEqual({
|
||||||
|
volume: 7.834176,
|
||||||
|
surface: 100.774998,
|
||||||
|
addedVolume: 7.975798,
|
||||||
|
removedVolume: 0.141622,
|
||||||
|
matchingCellsPct: 99.8,
|
||||||
|
groundNonMatchingPct: 0.2,
|
||||||
|
ceilNonMatchingPct: 0.0,
|
||||||
|
avgNeighborsCurrent: 8.0,
|
||||||
|
avgNeighborsRef: 8.0,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('leve une erreur explicite si un champ attendu est absent', async () => {
|
||||||
|
await fs.writeFile(tmpFile, 'contenu inattendu sans les champs voulus', 'utf-8');
|
||||||
|
await expect(service.parseVolumeReportFile(tmpFile)).rejects.toThrow(/Champ introuvable/);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('findLatestFile', () => {
|
||||||
|
let dir: string;
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
dir = await fs.mkdtemp(path.join(os.tmpdir(), 'cc-runner-test-'));
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
await fs.rm(dir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('trouve un fichier ecrit apres coup dans un sous-dossier (CloudCompare ecrit relativement au nuage charge, pas au cwd)', async () => {
|
||||||
|
const sub = path.join(dir, 'levels');
|
||||||
|
await fs.mkdir(sub);
|
||||||
|
const before = Date.now();
|
||||||
|
await new Promise((r) => setTimeout(r, 10));
|
||||||
|
const target = path.join(sub, 'VolumeCalculationReport_2026-01-01_00h00.txt');
|
||||||
|
await fs.writeFile(target, 'contenu', 'utf-8');
|
||||||
|
|
||||||
|
const found = await service.findLatestFile(dir, { prefix: 'VolumeCalculationReport', suffix: '.txt', after: before });
|
||||||
|
expect(found).toBe(target);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ignore les fichiers plus anciens que "after"', async () => {
|
||||||
|
const old = path.join(dir, 'old_report.txt');
|
||||||
|
await fs.writeFile(old, 'contenu', 'utf-8');
|
||||||
|
const after = Date.now() + 5000;
|
||||||
|
const found = await service.findLatestFile(dir, { suffix: '.txt', after });
|
||||||
|
expect(found).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
141
src/cloudcompare/cc-runner.service.ts
Normal file
141
src/cloudcompare/cc-runner.service.ts
Normal file
|
|
@ -0,0 +1,141 @@
|
||||||
|
import { Injectable, Logger } from '@nestjs/common';
|
||||||
|
import { spawn } from 'child_process';
|
||||||
|
import * as fs from 'fs/promises';
|
||||||
|
import * as path from 'path';
|
||||||
|
import { config } from '../config';
|
||||||
|
|
||||||
|
export interface CcResult {
|
||||||
|
stdout: string;
|
||||||
|
exitCode: number | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VolumeReport {
|
||||||
|
volume: number;
|
||||||
|
surface: number;
|
||||||
|
addedVolume: number;
|
||||||
|
removedVolume: number;
|
||||||
|
matchingCellsPct: number;
|
||||||
|
groundNonMatchingPct: number;
|
||||||
|
ceilNonMatchingPct: number;
|
||||||
|
avgNeighborsCurrent: number | null;
|
||||||
|
avgNeighborsRef: number | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Wrapper bas niveau autour du binaire CloudCompare en ligne de commande.
|
||||||
|
* Chaque appel est un process independant (le CLI CloudCompare fonctionne comme
|
||||||
|
* une machine a etats sur les entites chargees pendant UN SEUL appel : on ne
|
||||||
|
* peut pas "continuer" une session precedente).
|
||||||
|
*/
|
||||||
|
@Injectable()
|
||||||
|
export class CcRunnerService {
|
||||||
|
private readonly logger = new Logger(CcRunnerService.name);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* autoSave=true est necessaire pour -VOLUME : CloudCompare ne genere le rapport
|
||||||
|
* "VolumeCalculationReport_*.txt" que si l'auto-sauvegarde est active (verifie empiriquement,
|
||||||
|
* non documente). -SS / -SAVE_CLOUDS / -RASTERIZE fonctionnent en revanche correctement avec
|
||||||
|
* autoSave=false (evite de polluer le dossier de travail avec des fichiers intermediaires).
|
||||||
|
*/
|
||||||
|
async run(args: string[], cwd: string, opts: { autoSave?: boolean } = {}): Promise<CcResult> {
|
||||||
|
await fs.mkdir(cwd, { recursive: true });
|
||||||
|
const fullArgs = ['-SILENT', '-AUTO_SAVE', opts.autoSave ? 'ON' : 'OFF', ...args];
|
||||||
|
this.logger.debug(`CloudCompare ${fullArgs.join(' ')} (cwd=${cwd})`);
|
||||||
|
|
||||||
|
return new Promise<CcResult>((resolve, reject) => {
|
||||||
|
const child = spawn(config.cloudCompareBin, fullArgs, {
|
||||||
|
cwd,
|
||||||
|
windowsHide: true,
|
||||||
|
env: process.env,
|
||||||
|
});
|
||||||
|
|
||||||
|
let stdout = '';
|
||||||
|
const timer = setTimeout(() => {
|
||||||
|
child.kill('SIGKILL');
|
||||||
|
reject(new Error(`CloudCompare timeout apres ${config.ccTimeoutMs}ms (args: ${fullArgs.join(' ')})`));
|
||||||
|
}, config.ccTimeoutMs);
|
||||||
|
|
||||||
|
child.stdout?.on('data', (d) => (stdout += d.toString()));
|
||||||
|
child.stderr?.on('data', (d) => (stdout += d.toString()));
|
||||||
|
|
||||||
|
child.on('error', (err) => {
|
||||||
|
clearTimeout(timer);
|
||||||
|
reject(new Error(`Impossible de lancer CloudCompare (${config.cloudCompareBin}): ${err.message}`));
|
||||||
|
});
|
||||||
|
|
||||||
|
child.on('close', (code) => {
|
||||||
|
clearTimeout(timer);
|
||||||
|
resolve({ stdout, exitCode: code });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Cherche "Found one cloud with N points" (premiere occurrence = nuage juste charge par -O) */
|
||||||
|
parseLoadedPointCount(log: string): number | null {
|
||||||
|
const m = log.match(/Found one cloud with (\d+) points/);
|
||||||
|
return m ? Number(m[1]) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Cherche "Result: N points" (sortie de -SS SPATIAL/RANDOM) */
|
||||||
|
parseSubsampleResult(log: string): number | null {
|
||||||
|
const m = log.match(/\[SUBSAMPLE\][\s\S]*?Result:\s*(\d+)\s*points/);
|
||||||
|
return m ? Number(m[1]) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Trouve le fichier le plus recent correspondant a un prefixe/suffixe, apres un instant donne.
|
||||||
|
* Recherche recursive : CloudCompare ecrit les rapports/rasters relativement au dossier du
|
||||||
|
* nuage charge via -O (ex: "levels/"), pas forcement au cwd du process.
|
||||||
|
*/
|
||||||
|
async findLatestFile(rootDir: string, opts: { prefix?: string; suffix?: string; after: number }): Promise<string | null> {
|
||||||
|
const candidates: Array<{ full: string; mtime: number }> = [];
|
||||||
|
|
||||||
|
const walk = async (dir: string): Promise<void> => {
|
||||||
|
const entries = await fs.readdir(dir, { withFileTypes: true });
|
||||||
|
for (const entry of entries) {
|
||||||
|
const full = path.join(dir, entry.name);
|
||||||
|
if (entry.isDirectory()) {
|
||||||
|
await walk(full);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (opts.prefix && !entry.name.startsWith(opts.prefix)) continue;
|
||||||
|
if (opts.suffix && !entry.name.endsWith(opts.suffix)) continue;
|
||||||
|
const stat = await fs.stat(full);
|
||||||
|
const mtime = stat.mtimeMs;
|
||||||
|
if (mtime < opts.after - 2000) continue; // marge de securite horloge
|
||||||
|
candidates.push({ full, mtime });
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
await walk(rootDir);
|
||||||
|
if (candidates.length === 0) return null;
|
||||||
|
candidates.sort((a, b) => b.mtime - a.mtime);
|
||||||
|
return candidates[0].full;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Parse le fichier texte "VolumeCalculationReport_*.txt" genere par -VOLUME */
|
||||||
|
async parseVolumeReportFile(filePath: string): Promise<VolumeReport> {
|
||||||
|
const text = await fs.readFile(filePath, 'utf-8');
|
||||||
|
const num = (re: RegExp): number => {
|
||||||
|
const m = text.match(re);
|
||||||
|
if (!m) throw new Error(`Champ introuvable dans le rapport de volume (${re}): ${filePath}`);
|
||||||
|
return Number(m[1]);
|
||||||
|
};
|
||||||
|
const numOpt = (re: RegExp): number | null => {
|
||||||
|
const m = text.match(re);
|
||||||
|
return m ? Number(m[1]) : null;
|
||||||
|
};
|
||||||
|
|
||||||
|
return {
|
||||||
|
volume: num(/Volume:\s*([-\d.eE]+)/),
|
||||||
|
surface: num(/Surface:\s*([-\d.eE]+)/),
|
||||||
|
addedVolume: num(/Added volume:\s*\(\+\)\s*([-\d.eE]+)/),
|
||||||
|
removedVolume: num(/Removed volume:\s*\(-\)\s*([-\d.eE]+)/),
|
||||||
|
matchingCellsPct: num(/Matching cells:\s*([-\d.eE]+)\s*%/),
|
||||||
|
groundNonMatchingPct: num(/ground\s*=\s*([-\d.eE]+)\s*%/),
|
||||||
|
ceilNonMatchingPct: num(/ceil\s*=\s*([-\d.eE]+)\s*%/),
|
||||||
|
avgNeighborsCurrent: numOpt(/Average neighbors per cell:\s*([-\d.eE]+)\s*\//),
|
||||||
|
avgNeighborsRef: numOpt(/Average neighbors per cell:.*\/\s*([-\d.eE]+)/),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
8
src/cloudcompare/cloudcompare.module.ts
Normal file
8
src/cloudcompare/cloudcompare.module.ts
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
import { Module } from '@nestjs/common';
|
||||||
|
import { CcRunnerService } from './cc-runner.service';
|
||||||
|
|
||||||
|
@Module({
|
||||||
|
providers: [CcRunnerService],
|
||||||
|
exports: [CcRunnerService],
|
||||||
|
})
|
||||||
|
export class CloudCompareModule {}
|
||||||
36
src/config.ts
Normal file
36
src/config.ts
Normal file
|
|
@ -0,0 +1,36 @@
|
||||||
|
import * as path from 'path';
|
||||||
|
|
||||||
|
function envInt(name: string, fallback: number): number {
|
||||||
|
const v = process.env[name];
|
||||||
|
if (!v) return fallback;
|
||||||
|
const n = Number(v);
|
||||||
|
return Number.isFinite(n) ? n : fallback;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const config = {
|
||||||
|
port: envInt('PORT', 3000),
|
||||||
|
dataDir: process.env.DATA_DIR ?? path.resolve(process.cwd(), 'data'),
|
||||||
|
cloudCompareBin: process.env.CLOUDCOMPARE_BIN ?? 'CloudCompare',
|
||||||
|
pythonBin: process.env.PYTHON_BIN ?? 'python3',
|
||||||
|
// Pipeline constants (voir plan: facteur 2, 5 etapes)
|
||||||
|
decimationFactor: 2,
|
||||||
|
decimationSteps: 5,
|
||||||
|
// Nombre max de points echantillonnes pour estimer la distance moyenne au plus proche voisin
|
||||||
|
statsSampleCap: envInt('STATS_SAMPLE_CAP', 50000),
|
||||||
|
// Le pas de grille utilise pour TOUS les calculs de volume (fixe, derive du pas spatial initial)
|
||||||
|
// afin que les 6 niveaux restent comparables entre eux.
|
||||||
|
gridStepMultiplier: 2,
|
||||||
|
// En dessous de ce nombre de points, on arrete la decimation progressive (cloud devenu inexploitable)
|
||||||
|
minPointsToContinue: 25,
|
||||||
|
// Seuil d'alerte robustesse : si "matching cells %" du rapport CloudCompare passe sous ce seuil,
|
||||||
|
// le resultat est signale comme peu fiable dans l'UI.
|
||||||
|
matchingCellsWarnThreshold: 90,
|
||||||
|
// Taille max d'upload (octets)
|
||||||
|
maxUploadBytes: envInt('MAX_UPLOAD_MB', 2048) * 1024 * 1024,
|
||||||
|
// Timeout par appel CloudCompare CLI (ms)
|
||||||
|
ccTimeoutMs: envInt('CC_TIMEOUT_MS', 10 * 60 * 1000),
|
||||||
|
};
|
||||||
|
|
||||||
|
export const uploadsDir = () => path.join(config.dataDir, 'uploads');
|
||||||
|
export const resultsDir = () => path.join(config.dataDir, 'results');
|
||||||
|
export const dbPath = () => path.join(config.dataDir, 'app.db');
|
||||||
14
src/main.ts
Normal file
14
src/main.ts
Normal file
|
|
@ -0,0 +1,14 @@
|
||||||
|
import 'reflect-metadata';
|
||||||
|
import { NestFactory } from '@nestjs/core';
|
||||||
|
import { Logger } from '@nestjs/common';
|
||||||
|
import { AppModule } from './app.module';
|
||||||
|
import { config } from './config';
|
||||||
|
|
||||||
|
async function bootstrap() {
|
||||||
|
const app = await NestFactory.create(AppModule, { bodyParser: true });
|
||||||
|
app.enableCors();
|
||||||
|
await app.listen(config.port, '0.0.0.0');
|
||||||
|
new Logger('Bootstrap').log(`Application demarree sur http://0.0.0.0:${config.port} (data dir: ${config.dataDir})`);
|
||||||
|
}
|
||||||
|
|
||||||
|
bootstrap();
|
||||||
11
src/pipeline/pipeline.module.ts
Normal file
11
src/pipeline/pipeline.module.ts
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
import { Module } from '@nestjs/common';
|
||||||
|
import { CloudCompareModule } from '../cloudcompare/cloudcompare.module';
|
||||||
|
import { PyHelperModule } from '../pyhelper/pyhelper.module';
|
||||||
|
import { PipelineService } from './pipeline.service';
|
||||||
|
|
||||||
|
@Module({
|
||||||
|
imports: [CloudCompareModule, PyHelperModule],
|
||||||
|
providers: [PipelineService],
|
||||||
|
exports: [PipelineService],
|
||||||
|
})
|
||||||
|
export class PipelineModule {}
|
||||||
447
src/pipeline/pipeline.service.ts
Normal file
447
src/pipeline/pipeline.service.ts
Normal file
|
|
@ -0,0 +1,447 @@
|
||||||
|
import { Injectable, Logger } from '@nestjs/common';
|
||||||
|
import * as fs from 'fs/promises';
|
||||||
|
import * as path from 'path';
|
||||||
|
import { config } from '../config';
|
||||||
|
import { CcRunnerService } from '../cloudcompare/cc-runner.service';
|
||||||
|
import { PyHelperService } from '../pyhelper/pyhelper.service';
|
||||||
|
import { LevelResult, RunParamsInput, RunReport } from './types';
|
||||||
|
|
||||||
|
export function sanitizeBaseName(filename: string): string {
|
||||||
|
const base = filename.replace(/\.[^./\\]+$/, '');
|
||||||
|
return base.replace(/[^a-zA-Z0-9_-]+/g, '_').slice(0, 80) || 'nuage';
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Orchestration complete d'un run :
|
||||||
|
* 1) statistiques (pas spatial initial, plan de reference Z)
|
||||||
|
* 2) niveau 0 = nuage complet -> volume + image
|
||||||
|
* 3) niveaux 1..N = decimation spatiale progressive (facteur 2, chainee depuis le niveau precedent)
|
||||||
|
* 4) assemblage des nuages exploitables dans un seul .bin
|
||||||
|
* 5) rapport CSV/JSON + graphiques de synthese
|
||||||
|
*/
|
||||||
|
@Injectable()
|
||||||
|
export class PipelineService {
|
||||||
|
private readonly logger = new Logger(PipelineService.name);
|
||||||
|
|
||||||
|
constructor(
|
||||||
|
private readonly cc: CcRunnerService,
|
||||||
|
private readonly py: PyHelperService,
|
||||||
|
) {}
|
||||||
|
|
||||||
|
async execute(
|
||||||
|
runId: string,
|
||||||
|
inputPath: string,
|
||||||
|
originalFilename: string,
|
||||||
|
workDir: string,
|
||||||
|
params: RunParamsInput,
|
||||||
|
onProgress: (msg: string) => void,
|
||||||
|
): Promise<RunReport> {
|
||||||
|
const levelsDir = path.join(workDir, 'levels');
|
||||||
|
const imagesDir = path.join(workDir, 'images');
|
||||||
|
const rawDir = path.join(workDir, 'raw');
|
||||||
|
await fs.mkdir(levelsDir, { recursive: true });
|
||||||
|
await fs.mkdir(imagesDir, { recursive: true });
|
||||||
|
await fs.mkdir(rawDir, { recursive: true });
|
||||||
|
|
||||||
|
// ---------- 0) Normalisation du format d'entree ----------
|
||||||
|
// Le paquet apt CloudCompare (Debian, utilise dans l'image Docker) ne fournit pas le plugin
|
||||||
|
// LAS/LAZ (seulement "Core I/O"). On convertit donc LAS/LAZ/COPC en ASCII XYZ via laspy avant
|
||||||
|
// de transmettre le nuage a CloudCompare, qui lit nativement l'ASCII sans plugin.
|
||||||
|
let normalizedInputPath = inputPath;
|
||||||
|
const lowerInput = inputPath.toLowerCase();
|
||||||
|
if (lowerInput.endsWith('.las') || lowerInput.endsWith('.laz')) {
|
||||||
|
onProgress('Conversion du nuage source (LAS/LAZ/COPC) en ASCII (CloudCompare ne lit pas le LAS nativement dans ce conteneur)...');
|
||||||
|
const xyzPath = path.join(workDir, 'input_converted.xyz');
|
||||||
|
await this.py.convertLasToXyz(inputPath, xyzPath);
|
||||||
|
normalizedInputPath = xyzPath;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 1) Statistiques ----------
|
||||||
|
onProgress('Estimation du pas spatial initial (echantillonnage + KD-tree)...');
|
||||||
|
const t0 = Date.now();
|
||||||
|
const sampleRes = await this.cc.run(
|
||||||
|
['-O', normalizedInputPath, '-SS', 'RANDOM', String(config.statsSampleCap), '-C_EXPORT_FMT', 'ASC', '-PREC', '6', '-SAVE_CLOUDS', 'FILE', 'sample.xyz'],
|
||||||
|
workDir,
|
||||||
|
);
|
||||||
|
const totalPointCount = this.cc.parseLoadedPointCount(sampleRes.stdout);
|
||||||
|
if (totalPointCount === null) {
|
||||||
|
throw new Error(`Impossible de lire le nuage d'entree. Sortie CloudCompare:\n${sampleRes.stdout.slice(-2000)}`);
|
||||||
|
}
|
||||||
|
const samplePath = path.join(workDir, 'sample.xyz');
|
||||||
|
await this.assertExists(samplePath, 'echantillonnage du nuage (sample.xyz)', sampleRes.stdout);
|
||||||
|
const stats = await this.py.computeStats(samplePath);
|
||||||
|
|
||||||
|
const initialStep = params.initialStepOverride ?? stats.median_nn;
|
||||||
|
const zref = params.zrefOverride ?? stats.bbox.zmin;
|
||||||
|
const gridStep = initialStep * config.gridStepMultiplier;
|
||||||
|
|
||||||
|
onProgress(
|
||||||
|
`Pas initial=${initialStep.toFixed(5)}m (${params.initialStepOverride ? 'override' : 'auto, mediane NN'}), ` +
|
||||||
|
`grille volume=${gridStep.toFixed(5)}m, Zref=${zref.toFixed(4)} (${params.zrefOverride ? 'override' : 'auto, Zmin echantillon'}). ` +
|
||||||
|
`[${Date.now() - t0}ms]`,
|
||||||
|
);
|
||||||
|
|
||||||
|
const levels: LevelResult[] = [];
|
||||||
|
|
||||||
|
// ---------- 2) Niveau 0 : nuage complet ----------
|
||||||
|
onProgress('Niveau 0 (nuage complet) : conversion .bin + calcul de volume...');
|
||||||
|
const level0 = await this.computeLevel({
|
||||||
|
level: 0,
|
||||||
|
spatialStep: null,
|
||||||
|
sourceBinOrOriginal: normalizedInputPath,
|
||||||
|
isFirstConversion: true,
|
||||||
|
levelsDir,
|
||||||
|
imagesDir,
|
||||||
|
rawDir,
|
||||||
|
workDir,
|
||||||
|
gridStep,
|
||||||
|
zref,
|
||||||
|
refPointCount: totalPointCount,
|
||||||
|
onProgress,
|
||||||
|
});
|
||||||
|
levels.push(level0);
|
||||||
|
|
||||||
|
// ---------- 3) Niveaux 1..N : decimation progressive ----------
|
||||||
|
let prevBinAbs = level0.binFile ? path.join(workDir, level0.binFile) : null;
|
||||||
|
let prevPointCount = level0.pointCount;
|
||||||
|
let stopped = level0.status !== 'ok';
|
||||||
|
let stopReason = level0.status !== 'ok' ? 'le niveau 0 a echoue' : '';
|
||||||
|
|
||||||
|
for (let i = 1; i <= config.decimationSteps; i++) {
|
||||||
|
if (stopped || !prevBinAbs) {
|
||||||
|
levels.push(this.skippedLevel(i, stopReason));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (prevPointCount < config.minPointsToContinue) {
|
||||||
|
stopReason = `nuage trop clairseme apres le niveau ${i - 1} (${prevPointCount} points < seuil ${config.minPointsToContinue})`;
|
||||||
|
levels.push(this.skippedLevel(i, stopReason));
|
||||||
|
stopped = true;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const spatialStep = initialStep * Math.pow(config.decimationFactor, i);
|
||||||
|
onProgress(`Niveau ${i} : decimation spatiale (pas=${spatialStep.toFixed(5)}m) depuis le niveau ${i - 1}...`);
|
||||||
|
|
||||||
|
try {
|
||||||
|
const lvl = await this.computeLevel({
|
||||||
|
level: i,
|
||||||
|
spatialStep,
|
||||||
|
sourceBinOrOriginal: prevBinAbs,
|
||||||
|
isFirstConversion: false,
|
||||||
|
levelsDir,
|
||||||
|
imagesDir,
|
||||||
|
rawDir,
|
||||||
|
workDir,
|
||||||
|
gridStep,
|
||||||
|
zref,
|
||||||
|
refPointCount: totalPointCount,
|
||||||
|
onProgress,
|
||||||
|
});
|
||||||
|
levels.push(lvl);
|
||||||
|
if (lvl.status === 'ok') {
|
||||||
|
prevBinAbs = lvl.binFile ? path.join(workDir, lvl.binFile) : null;
|
||||||
|
prevPointCount = lvl.pointCount;
|
||||||
|
} else {
|
||||||
|
stopped = true;
|
||||||
|
stopReason = lvl.message ?? `echec au niveau ${i}`;
|
||||||
|
}
|
||||||
|
} catch (err: any) {
|
||||||
|
this.logger.error(`Niveau ${i} en erreur: ${err.message}`);
|
||||||
|
levels.push({
|
||||||
|
level: i,
|
||||||
|
spatialStep,
|
||||||
|
pointCount: 0,
|
||||||
|
pointRatio: 0,
|
||||||
|
volume: null,
|
||||||
|
surface: null,
|
||||||
|
addedVolume: null,
|
||||||
|
removedVolume: null,
|
||||||
|
matchingCellsPct: null,
|
||||||
|
groundNonMatchingPct: null,
|
||||||
|
ceilNonMatchingPct: null,
|
||||||
|
warn: true,
|
||||||
|
status: 'error',
|
||||||
|
message: err.message,
|
||||||
|
binFile: null,
|
||||||
|
imageFile: null,
|
||||||
|
computeTimeMs: 0,
|
||||||
|
});
|
||||||
|
stopped = true;
|
||||||
|
stopReason = err.message;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 4) Assemblage ----------
|
||||||
|
onProgress('Assemblage des nuages exploitables dans un seul fichier .bin...');
|
||||||
|
const okLevels = levels.filter((l) => l.status === 'ok' && l.binFile);
|
||||||
|
let mergedBinFile: string | null = null;
|
||||||
|
if (okLevels.length > 0) {
|
||||||
|
const baseName = sanitizeBaseName(originalFilename);
|
||||||
|
const mergedName = `${baseName}_all_levels.bin`;
|
||||||
|
const args: string[] = [];
|
||||||
|
for (const lvl of okLevels) args.push('-O', lvl.binFile!);
|
||||||
|
args.push('-C_EXPORT_FMT', 'BIN', '-SAVE_CLOUDS', 'ALL_AT_ONCE', 'FILE', mergedName);
|
||||||
|
const mergeRes = await this.cc.run(args, workDir);
|
||||||
|
const mergedAbs = path.join(workDir, mergedName);
|
||||||
|
await this.assertExists(mergedAbs, 'fusion des nuages', mergeRes.stdout);
|
||||||
|
mergedBinFile = mergedName;
|
||||||
|
}
|
||||||
|
|
||||||
|
const report: RunReport = {
|
||||||
|
runId,
|
||||||
|
originalFilename,
|
||||||
|
createdAt: new Date().toISOString(),
|
||||||
|
params: {
|
||||||
|
zref,
|
||||||
|
zrefSource: params.zrefOverride !== undefined ? 'override' : 'auto',
|
||||||
|
initialStep,
|
||||||
|
initialStepSource: params.initialStepOverride !== undefined ? 'override' : 'auto',
|
||||||
|
gridStep,
|
||||||
|
factor: config.decimationFactor,
|
||||||
|
steps: config.decimationSteps,
|
||||||
|
sampleStats: {
|
||||||
|
sampleCount: stats.count,
|
||||||
|
meanNn: stats.mean_nn,
|
||||||
|
medianNn: stats.median_nn,
|
||||||
|
bboxApprox: stats.bbox,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
levels,
|
||||||
|
mergedBinFile,
|
||||||
|
};
|
||||||
|
|
||||||
|
await fs.writeFile(path.join(workDir, 'report.json'), JSON.stringify(report, null, 2), 'utf-8');
|
||||||
|
await fs.writeFile(path.join(workDir, 'report.csv'), this.toCsv(report), 'utf-8');
|
||||||
|
|
||||||
|
// ---------- 5) Graphiques de synthese ----------
|
||||||
|
onProgress('Generation des graphiques de synthese...');
|
||||||
|
try {
|
||||||
|
await this.py.renderSummary(path.join(workDir, 'report.json'), imagesDir);
|
||||||
|
} catch (err: any) {
|
||||||
|
this.logger.warn(`Graphiques de synthese non generes: ${err.message}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return report;
|
||||||
|
}
|
||||||
|
|
||||||
|
private skippedLevel(level: number, reason: string): LevelResult {
|
||||||
|
return {
|
||||||
|
level,
|
||||||
|
spatialStep: null,
|
||||||
|
pointCount: 0,
|
||||||
|
pointRatio: 0,
|
||||||
|
volume: null,
|
||||||
|
surface: null,
|
||||||
|
addedVolume: null,
|
||||||
|
removedVolume: null,
|
||||||
|
matchingCellsPct: null,
|
||||||
|
groundNonMatchingPct: null,
|
||||||
|
ceilNonMatchingPct: null,
|
||||||
|
warn: true,
|
||||||
|
status: 'skipped',
|
||||||
|
message: reason,
|
||||||
|
binFile: null,
|
||||||
|
imageFile: null,
|
||||||
|
computeTimeMs: 0,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private async assertExists(p: string, what: string, ccLog: string) {
|
||||||
|
try {
|
||||||
|
await fs.access(p);
|
||||||
|
} catch {
|
||||||
|
throw new Error(`Etape "${what}" : fichier attendu introuvable (${p}). Sortie CloudCompare:\n${ccLog.slice(-2000)}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async computeLevel(opts: {
|
||||||
|
level: number;
|
||||||
|
spatialStep: number | null;
|
||||||
|
sourceBinOrOriginal: string;
|
||||||
|
isFirstConversion: boolean;
|
||||||
|
levelsDir: string;
|
||||||
|
imagesDir: string;
|
||||||
|
rawDir: string;
|
||||||
|
workDir: string;
|
||||||
|
gridStep: number;
|
||||||
|
zref: number;
|
||||||
|
refPointCount: number;
|
||||||
|
onProgress: (msg: string) => void;
|
||||||
|
}): Promise<LevelResult> {
|
||||||
|
const { level, spatialStep, workDir, gridStep, zref, refPointCount } = opts;
|
||||||
|
const t0 = Date.now();
|
||||||
|
let binRelPath: string;
|
||||||
|
let pointCount: number;
|
||||||
|
|
||||||
|
if (opts.isFirstConversion) {
|
||||||
|
// Niveau 0 : simple conversion du fichier d'entree en .bin canonique
|
||||||
|
binRelPath = path.posix.join('levels', 'L0_full.bin');
|
||||||
|
const res = await this.cc.run(['-O', opts.sourceBinOrOriginal, '-C_EXPORT_FMT', 'BIN', '-SAVE_CLOUDS', 'FILE', binRelPath], workDir);
|
||||||
|
const n = this.cc.parseLoadedPointCount(res.stdout);
|
||||||
|
if (n === null) {
|
||||||
|
return this.errorLevel(level, spatialStep, `Conversion niveau 0 echouee. Sortie:\n${res.stdout.slice(-1500)}`, t0);
|
||||||
|
}
|
||||||
|
pointCount = n;
|
||||||
|
await this.assertExists(path.join(workDir, binRelPath), 'conversion niveau 0', res.stdout);
|
||||||
|
} else {
|
||||||
|
// Niveaux 1..N : decimation spatiale depuis le niveau precedent
|
||||||
|
const stepStr = spatialStep!.toFixed(6);
|
||||||
|
binRelPath = path.posix.join('levels', `L${level}_step${stepStr}.bin`);
|
||||||
|
const res = await this.cc.run(
|
||||||
|
['-O', opts.sourceBinOrOriginal, '-SS', 'SPATIAL', stepStr, '-C_EXPORT_FMT', 'BIN', '-SAVE_CLOUDS', 'FILE', binRelPath],
|
||||||
|
workDir,
|
||||||
|
);
|
||||||
|
const n = this.cc.parseSubsampleResult(res.stdout);
|
||||||
|
if (n === null || n === 0) {
|
||||||
|
return {
|
||||||
|
level,
|
||||||
|
spatialStep,
|
||||||
|
pointCount: 0,
|
||||||
|
pointRatio: 0,
|
||||||
|
volume: null,
|
||||||
|
surface: null,
|
||||||
|
addedVolume: null,
|
||||||
|
removedVolume: null,
|
||||||
|
matchingCellsPct: null,
|
||||||
|
groundNonMatchingPct: null,
|
||||||
|
ceilNonMatchingPct: null,
|
||||||
|
warn: true,
|
||||||
|
status: 'skipped',
|
||||||
|
message: `decimation spatiale n'a produit aucun point exploitable (pas=${stepStr}m)`,
|
||||||
|
binFile: null,
|
||||||
|
imageFile: null,
|
||||||
|
computeTimeMs: Date.now() - t0,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
pointCount = n;
|
||||||
|
await this.assertExists(path.join(workDir, binRelPath), `decimation niveau ${level}`, res.stdout);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Volume (necessite AUTO_SAVE ON, sinon CloudCompare ne genere pas le rapport texte - voir cc-runner.service.ts)
|
||||||
|
const beforeVolume = Date.now();
|
||||||
|
const volRes = await this.cc.run(
|
||||||
|
['-O', binRelPath, '-VOLUME', '-GRID_STEP', String(gridStep), '-CONST_HEIGHT', String(zref)],
|
||||||
|
workDir,
|
||||||
|
{ autoSave: true },
|
||||||
|
);
|
||||||
|
const reportFile = await this.cc.findLatestFile(workDir, { prefix: 'VolumeCalculationReport', suffix: '.txt', after: beforeVolume });
|
||||||
|
if (!reportFile) {
|
||||||
|
return this.errorLevel(level, spatialStep, `Rapport de volume introuvable au niveau ${level}. Sortie:\n${volRes.stdout.slice(-1500)}`, t0, binRelPath);
|
||||||
|
}
|
||||||
|
const vol = await this.cc.parseVolumeReportFile(reportFile);
|
||||||
|
const rawReportDest = path.join(opts.rawDir, `L${level}_volume_report.txt`);
|
||||||
|
await fs.rename(reportFile, rawReportDest);
|
||||||
|
|
||||||
|
// AUTO_SAVE ON sauvegarde aussi une grille "*_HEIGHT_DIFFERENCE_*.bin" qu'on ne veut pas garder
|
||||||
|
const strayGrid = await this.cc.findLatestFile(workDir, { suffix: '.bin', after: beforeVolume });
|
||||||
|
if (strayGrid && strayGrid !== path.join(workDir, binRelPath)) {
|
||||||
|
await fs.unlink(strayGrid).catch(() => undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Carte de hauteur : export ASCII du nuage + binning numpy cote Python.
|
||||||
|
// (Le paquet apt CloudCompare, utilise dans l'image Docker, plante sur l'export GeoTIFF natif
|
||||||
|
// -RASTERIZE -OUTPUT_RASTER_Z faute de support GDAL compile - on reconstruit donc la grille
|
||||||
|
// nous-memes a partir d'un export ASCII, qui lui fonctionne sans plugin.)
|
||||||
|
let imageRelPath: string | null = null;
|
||||||
|
try {
|
||||||
|
const xyzExportRel = path.posix.join('raw', `L${level}_points.xyz`);
|
||||||
|
await this.cc.run(['-O', binRelPath, '-C_EXPORT_FMT', 'ASC', '-PREC', '6', '-SAVE_CLOUDS', 'FILE', xyzExportRel], workDir);
|
||||||
|
const xyzAbs = path.join(workDir, xyzExportRel);
|
||||||
|
await this.assertExists(xyzAbs, `export ASCII niveau ${level}`, '');
|
||||||
|
const pngName = `L${level}_heightmap.png`;
|
||||||
|
const pngAbs = path.join(opts.imagesDir, pngName);
|
||||||
|
const title = spatialStep === null ? `Niveau 0 (complet, ${pointCount} pts)` : `Niveau ${level} (pas=${spatialStep.toFixed(4)}m, ${pointCount} pts)`;
|
||||||
|
await this.py.renderHeightmap(xyzAbs, gridStep, pngAbs, title);
|
||||||
|
imageRelPath = path.posix.join('images', pngName);
|
||||||
|
} catch (err: any) {
|
||||||
|
this.logger.warn(`Image niveau ${level} non generee: ${err.message}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const warn = vol.matchingCellsPct < config.matchingCellsWarnThreshold;
|
||||||
|
|
||||||
|
return {
|
||||||
|
level,
|
||||||
|
spatialStep,
|
||||||
|
pointCount,
|
||||||
|
pointRatio: refPointCount > 0 ? pointCount / refPointCount : 0,
|
||||||
|
volume: vol.volume,
|
||||||
|
surface: vol.surface,
|
||||||
|
addedVolume: vol.addedVolume,
|
||||||
|
removedVolume: vol.removedVolume,
|
||||||
|
matchingCellsPct: vol.matchingCellsPct,
|
||||||
|
groundNonMatchingPct: vol.groundNonMatchingPct,
|
||||||
|
ceilNonMatchingPct: vol.ceilNonMatchingPct,
|
||||||
|
warn,
|
||||||
|
status: 'ok',
|
||||||
|
binFile: binRelPath,
|
||||||
|
imageFile: imageRelPath,
|
||||||
|
computeTimeMs: Date.now() - t0,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private errorLevel(level: number, spatialStep: number | null, message: string, t0: number, binFile: string | null = null): LevelResult {
|
||||||
|
return {
|
||||||
|
level,
|
||||||
|
spatialStep,
|
||||||
|
pointCount: 0,
|
||||||
|
pointRatio: 0,
|
||||||
|
volume: null,
|
||||||
|
surface: null,
|
||||||
|
addedVolume: null,
|
||||||
|
removedVolume: null,
|
||||||
|
matchingCellsPct: null,
|
||||||
|
groundNonMatchingPct: null,
|
||||||
|
ceilNonMatchingPct: null,
|
||||||
|
warn: true,
|
||||||
|
status: 'error',
|
||||||
|
message,
|
||||||
|
binFile,
|
||||||
|
imageFile: null,
|
||||||
|
computeTimeMs: Date.now() - t0,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private toCsv(report: RunReport): string {
|
||||||
|
return reportToCsv(report);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function reportToCsv(report: RunReport): string {
|
||||||
|
const headers = [
|
||||||
|
'level',
|
||||||
|
'spatialStep',
|
||||||
|
'pointCount',
|
||||||
|
'pointRatio',
|
||||||
|
'volume',
|
||||||
|
'surface',
|
||||||
|
'addedVolume',
|
||||||
|
'removedVolume',
|
||||||
|
'matchingCellsPct',
|
||||||
|
'groundNonMatchingPct',
|
||||||
|
'ceilNonMatchingPct',
|
||||||
|
'status',
|
||||||
|
'warn',
|
||||||
|
'computeTimeMs',
|
||||||
|
'message',
|
||||||
|
];
|
||||||
|
const rows = report.levels.map((l) =>
|
||||||
|
[
|
||||||
|
l.level,
|
||||||
|
l.spatialStep ?? '',
|
||||||
|
l.pointCount,
|
||||||
|
l.pointRatio,
|
||||||
|
l.volume ?? '',
|
||||||
|
l.surface ?? '',
|
||||||
|
l.addedVolume ?? '',
|
||||||
|
l.removedVolume ?? '',
|
||||||
|
l.matchingCellsPct ?? '',
|
||||||
|
l.groundNonMatchingPct ?? '',
|
||||||
|
l.ceilNonMatchingPct ?? '',
|
||||||
|
l.status,
|
||||||
|
l.warn,
|
||||||
|
l.computeTimeMs,
|
||||||
|
(l.message ?? '').replace(/[\r\n,]+/g, ' '),
|
||||||
|
].join(','),
|
||||||
|
);
|
||||||
|
return [headers.join(','), ...rows].join('\n') + '\n';
|
||||||
|
}
|
||||||
103
src/pipeline/pipeline.util.spec.ts
Normal file
103
src/pipeline/pipeline.util.spec.ts
Normal file
|
|
@ -0,0 +1,103 @@
|
||||||
|
import { reportToCsv, sanitizeBaseName } from './pipeline.service';
|
||||||
|
import { RunReport } from './types';
|
||||||
|
|
||||||
|
describe('sanitizeBaseName', () => {
|
||||||
|
it('retire l\'extension et les caracteres non surs', () => {
|
||||||
|
expect(sanitizeBaseName('mon nuage (2026) v2.las')).toBe('mon_nuage_2026_v2');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('gere le double-extension .copc.laz (ne retire que le dernier ".laz")', () => {
|
||||||
|
expect(sanitizeBaseName('zone_chantier.copc.laz')).toBe('zone_chantier_copc');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('retombe sur un nom par defaut si le nom de base est vide apres extraction de l\'extension', () => {
|
||||||
|
expect(sanitizeBaseName('.las')).toBe('nuage');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('tronque a 80 caracteres', () => {
|
||||||
|
const long = 'a'.repeat(200) + '.las';
|
||||||
|
expect(sanitizeBaseName(long).length).toBe(80);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('reportToCsv', () => {
|
||||||
|
const baseReport: RunReport = {
|
||||||
|
runId: 'run-1',
|
||||||
|
originalFilename: 'test.las',
|
||||||
|
createdAt: '2026-01-01T00:00:00.000Z',
|
||||||
|
params: {
|
||||||
|
zref: 100,
|
||||||
|
zrefSource: 'auto',
|
||||||
|
initialStep: 0.02,
|
||||||
|
initialStepSource: 'auto',
|
||||||
|
gridStep: 0.04,
|
||||||
|
factor: 2,
|
||||||
|
steps: 1,
|
||||||
|
sampleStats: { sampleCount: 100, meanNn: 0.02, medianNn: 0.02, bboxApprox: { xmin: 0, xmax: 1, ymin: 0, ymax: 1, zmin: 100, zmax: 101 } },
|
||||||
|
},
|
||||||
|
levels: [
|
||||||
|
{
|
||||||
|
level: 0,
|
||||||
|
spatialStep: null,
|
||||||
|
pointCount: 1000,
|
||||||
|
pointRatio: 1,
|
||||||
|
volume: 12.5,
|
||||||
|
surface: 100,
|
||||||
|
addedVolume: 12.5,
|
||||||
|
removedVolume: 0,
|
||||||
|
matchingCellsPct: 99,
|
||||||
|
groundNonMatchingPct: 1,
|
||||||
|
ceilNonMatchingPct: 0,
|
||||||
|
warn: false,
|
||||||
|
status: 'ok',
|
||||||
|
binFile: 'levels/L0_full.bin',
|
||||||
|
imageFile: 'images/L0_heightmap.png',
|
||||||
|
computeTimeMs: 500,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
level: 1,
|
||||||
|
spatialStep: 0.04,
|
||||||
|
pointCount: 0,
|
||||||
|
pointRatio: 0,
|
||||||
|
volume: null,
|
||||||
|
surface: null,
|
||||||
|
addedVolume: null,
|
||||||
|
removedVolume: null,
|
||||||
|
matchingCellsPct: null,
|
||||||
|
groundNonMatchingPct: null,
|
||||||
|
ceilNonMatchingPct: null,
|
||||||
|
warn: true,
|
||||||
|
status: 'skipped',
|
||||||
|
message: 'decimation, pas assez de points\navec une virgule, ici',
|
||||||
|
binFile: null,
|
||||||
|
imageFile: null,
|
||||||
|
computeTimeMs: 0,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
mergedBinFile: 'test_all_levels.bin',
|
||||||
|
};
|
||||||
|
|
||||||
|
it('genere un en-tete et une ligne par niveau', () => {
|
||||||
|
const csv = reportToCsv(baseReport);
|
||||||
|
const lines = csv.trim().split('\n');
|
||||||
|
expect(lines).toHaveLength(3); // header + 2 niveaux
|
||||||
|
expect(lines[0]).toBe(
|
||||||
|
'level,spatialStep,pointCount,pointRatio,volume,surface,addedVolume,removedVolume,matchingCellsPct,groundNonMatchingPct,ceilNonMatchingPct,status,warn,computeTimeMs,message',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('remplace les sauts de ligne et virgules dans les messages pour ne pas casser le CSV', () => {
|
||||||
|
const csv = reportToCsv(baseReport);
|
||||||
|
const lines = csv.trim().split('\n');
|
||||||
|
expect(lines[2]).not.toContain('\n');
|
||||||
|
// le message ne doit contenir aucune virgule qui ajouterait une colonne supplementaire
|
||||||
|
const messageField = lines[2].split(',').slice(14).join(',');
|
||||||
|
expect(messageField).not.toMatch(/,/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('laisse les champs numeriques null en cellule vide plutot que "null"', () => {
|
||||||
|
const csv = reportToCsv(baseReport);
|
||||||
|
const lines = csv.trim().split('\n');
|
||||||
|
expect(lines[2]).not.toContain('null');
|
||||||
|
});
|
||||||
|
});
|
||||||
47
src/pipeline/types.ts
Normal file
47
src/pipeline/types.ts
Normal file
|
|
@ -0,0 +1,47 @@
|
||||||
|
export interface LevelResult {
|
||||||
|
level: number;
|
||||||
|
spatialStep: number | null; // null pour le niveau 0 (nuage complet, non decime)
|
||||||
|
pointCount: number;
|
||||||
|
pointRatio: number; // vs niveau 0
|
||||||
|
volume: number | null;
|
||||||
|
surface: number | null;
|
||||||
|
addedVolume: number | null;
|
||||||
|
removedVolume: number | null;
|
||||||
|
matchingCellsPct: number | null;
|
||||||
|
groundNonMatchingPct: number | null;
|
||||||
|
ceilNonMatchingPct: number | null;
|
||||||
|
warn: boolean;
|
||||||
|
status: 'ok' | 'skipped' | 'error';
|
||||||
|
message?: string;
|
||||||
|
binFile: string | null; // chemin relatif au dossier du run
|
||||||
|
imageFile: string | null; // chemin relatif au dossier du run
|
||||||
|
computeTimeMs: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface RunParamsInput {
|
||||||
|
zrefOverride?: number;
|
||||||
|
initialStepOverride?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface RunReport {
|
||||||
|
runId: string;
|
||||||
|
originalFilename: string;
|
||||||
|
createdAt: string;
|
||||||
|
params: {
|
||||||
|
zref: number;
|
||||||
|
zrefSource: 'override' | 'auto';
|
||||||
|
initialStep: number;
|
||||||
|
initialStepSource: 'override' | 'auto';
|
||||||
|
gridStep: number;
|
||||||
|
factor: number;
|
||||||
|
steps: number;
|
||||||
|
sampleStats: {
|
||||||
|
sampleCount: number;
|
||||||
|
meanNn: number;
|
||||||
|
medianNn: number;
|
||||||
|
bboxApprox: { xmin: number; xmax: number; ymin: number; ymax: number; zmin: number; zmax: number };
|
||||||
|
};
|
||||||
|
};
|
||||||
|
levels: LevelResult[];
|
||||||
|
mergedBinFile: string | null;
|
||||||
|
}
|
||||||
8
src/pyhelper/pyhelper.module.ts
Normal file
8
src/pyhelper/pyhelper.module.ts
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
import { Module } from '@nestjs/common';
|
||||||
|
import { PyHelperService } from './pyhelper.service';
|
||||||
|
|
||||||
|
@Module({
|
||||||
|
providers: [PyHelperService],
|
||||||
|
exports: [PyHelperService],
|
||||||
|
})
|
||||||
|
export class PyHelperModule {}
|
||||||
84
src/pyhelper/pyhelper.service.ts
Normal file
84
src/pyhelper/pyhelper.service.ts
Normal file
|
|
@ -0,0 +1,84 @@
|
||||||
|
import { Injectable, Logger } from '@nestjs/common';
|
||||||
|
import { spawn } from 'child_process';
|
||||||
|
import * as path from 'path';
|
||||||
|
import { config } from '../config';
|
||||||
|
|
||||||
|
const PYTHON_DIR = path.resolve(__dirname, '..', '..', 'python');
|
||||||
|
|
||||||
|
export interface CloudStats {
|
||||||
|
count: number;
|
||||||
|
mean_nn: number;
|
||||||
|
median_nn: number;
|
||||||
|
std_nn: number;
|
||||||
|
bbox: { xmin: number; xmax: number; ymin: number; ymax: number; zmin: number; zmax: number };
|
||||||
|
approximate: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Wrapper autour des scripts python/*.py (statistiques + generation d'images). */
|
||||||
|
@Injectable()
|
||||||
|
export class PyHelperService {
|
||||||
|
private readonly logger = new Logger(PyHelperService.name);
|
||||||
|
|
||||||
|
private async runJson(scriptRelPath: string, args: string[]): Promise<any> {
|
||||||
|
const scriptPath = path.join(PYTHON_DIR, scriptRelPath);
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const child = spawn(config.pythonBin, [scriptPath, ...args], { windowsHide: true });
|
||||||
|
let stdout = '';
|
||||||
|
let stderr = '';
|
||||||
|
child.stdout.on('data', (d) => (stdout += d.toString()));
|
||||||
|
child.stderr.on('data', (d) => (stderr += d.toString()));
|
||||||
|
child.on('error', (err) => reject(new Error(`Impossible de lancer python (${config.pythonBin}): ${err.message}`)));
|
||||||
|
child.on('close', (code) => {
|
||||||
|
if (stderr) this.logger.warn(`${scriptRelPath} stderr: ${stderr.trim()}`);
|
||||||
|
const trimmed = stdout.trim();
|
||||||
|
if (!trimmed) {
|
||||||
|
reject(new Error(`${scriptRelPath} n'a produit aucune sortie (code ${code}). stderr: ${stderr}`));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let parsed: any;
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(trimmed.split('\n').pop()!);
|
||||||
|
} catch {
|
||||||
|
reject(new Error(`Sortie JSON invalide de ${scriptRelPath}: ${trimmed}`));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (parsed.error) {
|
||||||
|
reject(new Error(`${scriptRelPath}: ${parsed.error}`));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
resolve(parsed);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
computeStats(xyzSamplePath: string): Promise<CloudStats> {
|
||||||
|
return this.runJson('stats_helper.py', ['--input', xyzSamplePath]);
|
||||||
|
}
|
||||||
|
|
||||||
|
renderHeightmap(xyzPath: string, gridStep: number, outputPngPath: string, title: string): Promise<void> {
|
||||||
|
return this.runJson('render_images.py', [
|
||||||
|
'heightmap',
|
||||||
|
'--input',
|
||||||
|
xyzPath,
|
||||||
|
'--gridstep',
|
||||||
|
String(gridStep),
|
||||||
|
'--output',
|
||||||
|
outputPngPath,
|
||||||
|
'--title',
|
||||||
|
title,
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
|
renderSummary(reportJsonPath: string, outDir: string): Promise<void> {
|
||||||
|
return this.runJson('render_images.py', ['summary', '--report', reportJsonPath, '--outdir', outDir]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Le paquet apt CloudCompare (Debian) ne fournit pas le plugin LAS/LAZ (seulement "Core I/O").
|
||||||
|
* On convertit donc les entrees LAS/LAZ/COPC en ASCII XYZ via laspy avant de les transmettre
|
||||||
|
* a CloudCompare, qui lit nativement le format ASCII sans plugin.
|
||||||
|
*/
|
||||||
|
async convertLasToXyz(inputPath: string, outputXyzPath: string): Promise<{ count: number }> {
|
||||||
|
return this.runJson('las_to_xyz.py', ['--input', inputPath, '--output', outputXyzPath]);
|
||||||
|
}
|
||||||
|
}
|
||||||
78
src/runs/database.service.ts
Normal file
78
src/runs/database.service.ts
Normal file
|
|
@ -0,0 +1,78 @@
|
||||||
|
import { Injectable, OnModuleInit } from '@nestjs/common';
|
||||||
|
import Database from 'better-sqlite3';
|
||||||
|
import * as fs from 'fs';
|
||||||
|
import { config, dbPath } from '../config';
|
||||||
|
|
||||||
|
export interface RunRow {
|
||||||
|
id: string;
|
||||||
|
original_filename: string;
|
||||||
|
status: 'pending' | 'running' | 'done' | 'error';
|
||||||
|
created_at: string;
|
||||||
|
finished_at: string | null;
|
||||||
|
params_json: string | null;
|
||||||
|
report_json: string | null;
|
||||||
|
log: string;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Injectable()
|
||||||
|
export class DatabaseService implements OnModuleInit {
|
||||||
|
private db!: Database.Database;
|
||||||
|
|
||||||
|
onModuleInit() {
|
||||||
|
fs.mkdirSync(config.dataDir, { recursive: true });
|
||||||
|
this.db = new Database(dbPath());
|
||||||
|
this.db.pragma('journal_mode = WAL');
|
||||||
|
this.db.exec(`
|
||||||
|
CREATE TABLE IF NOT EXISTS runs (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
original_filename TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
finished_at TEXT,
|
||||||
|
params_json TEXT,
|
||||||
|
report_json TEXT,
|
||||||
|
log TEXT NOT NULL DEFAULT '',
|
||||||
|
error TEXT
|
||||||
|
);
|
||||||
|
`);
|
||||||
|
}
|
||||||
|
|
||||||
|
insert(row: Pick<RunRow, 'id' | 'original_filename' | 'status' | 'created_at' | 'params_json'>) {
|
||||||
|
this.db
|
||||||
|
.prepare(
|
||||||
|
`INSERT INTO runs (id, original_filename, status, created_at, params_json, log) VALUES (?, ?, ?, ?, ?, '')`,
|
||||||
|
)
|
||||||
|
.run(row.id, row.original_filename, row.status, row.created_at, row.params_json);
|
||||||
|
}
|
||||||
|
|
||||||
|
updateStatus(id: string, status: RunRow['status']) {
|
||||||
|
this.db.prepare(`UPDATE runs SET status = ? WHERE id = ?`).run(status, id);
|
||||||
|
}
|
||||||
|
|
||||||
|
appendLog(id: string, line: string) {
|
||||||
|
this.db.prepare(`UPDATE runs SET log = log || ? WHERE id = ?`).run(`[${new Date().toISOString()}] ${line}\n`, id);
|
||||||
|
}
|
||||||
|
|
||||||
|
complete(id: string, reportJson: string) {
|
||||||
|
this.db
|
||||||
|
.prepare(`UPDATE runs SET status = 'done', report_json = ?, finished_at = ? WHERE id = ?`)
|
||||||
|
.run(reportJson, new Date().toISOString(), id);
|
||||||
|
}
|
||||||
|
|
||||||
|
fail(id: string, error: string) {
|
||||||
|
this.db.prepare(`UPDATE runs SET status = 'error', error = ?, finished_at = ? WHERE id = ?`).run(error, new Date().toISOString(), id);
|
||||||
|
}
|
||||||
|
|
||||||
|
get(id: string): RunRow | undefined {
|
||||||
|
return this.db.prepare(`SELECT * FROM runs WHERE id = ?`).get(id) as RunRow | undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
list(): RunRow[] {
|
||||||
|
return this.db.prepare(`SELECT * FROM runs ORDER BY created_at DESC`).all() as RunRow[];
|
||||||
|
}
|
||||||
|
|
||||||
|
remove(id: string) {
|
||||||
|
this.db.prepare(`DELETE FROM runs WHERE id = ?`).run(id);
|
||||||
|
}
|
||||||
|
}
|
||||||
138
src/runs/runs.controller.ts
Normal file
138
src/runs/runs.controller.ts
Normal file
|
|
@ -0,0 +1,138 @@
|
||||||
|
import {
|
||||||
|
BadRequestException,
|
||||||
|
Body,
|
||||||
|
Controller,
|
||||||
|
Delete,
|
||||||
|
Get,
|
||||||
|
NotFoundException,
|
||||||
|
Param,
|
||||||
|
Post,
|
||||||
|
Res,
|
||||||
|
UploadedFile,
|
||||||
|
UseInterceptors,
|
||||||
|
} from '@nestjs/common';
|
||||||
|
import { FileInterceptor } from '@nestjs/platform-express';
|
||||||
|
import type { Response } from 'express';
|
||||||
|
import archiver from 'archiver';
|
||||||
|
import * as fs from 'fs';
|
||||||
|
import * as fsp from 'fs/promises';
|
||||||
|
import * as path from 'path';
|
||||||
|
import { diskStorage } from 'multer';
|
||||||
|
import { config, uploadsDir } from '../config';
|
||||||
|
import { RunsService } from './runs.service';
|
||||||
|
|
||||||
|
const ALLOWED_EXT = ['.las', '.laz', '.copc.laz', '.bin'];
|
||||||
|
|
||||||
|
function isAllowedFilename(name: string): boolean {
|
||||||
|
const lower = name.toLowerCase();
|
||||||
|
return ALLOWED_EXT.some((ext) => lower.endsWith(ext));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Controller('api/runs')
|
||||||
|
export class RunsController {
|
||||||
|
constructor(private readonly runs: RunsService) {}
|
||||||
|
|
||||||
|
@Post()
|
||||||
|
@UseInterceptors(
|
||||||
|
FileInterceptor('file', {
|
||||||
|
storage: diskStorage({
|
||||||
|
destination: (_req, _file, cb) => {
|
||||||
|
const dir = path.join(uploadsDir(), 'incoming');
|
||||||
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
cb(null, dir);
|
||||||
|
},
|
||||||
|
filename: (_req, file, cb) => {
|
||||||
|
cb(null, `${Date.now()}_${Math.random().toString(36).slice(2)}__${file.originalname}`);
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
limits: { fileSize: config.maxUploadBytes },
|
||||||
|
fileFilter: (_req, file, cb) => {
|
||||||
|
if (!isAllowedFilename(file.originalname)) {
|
||||||
|
cb(new BadRequestException(`Format non supporte. Extensions acceptees: ${ALLOWED_EXT.join(', ')}`) as any, false);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
cb(null, true);
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
async create(@UploadedFile() file: Express.Multer.File, @Body() body: Record<string, string>) {
|
||||||
|
if (!file) throw new BadRequestException('Aucun fichier recu (champ "file" attendu)');
|
||||||
|
|
||||||
|
const zrefOverride = this.parseOptionalNumber(body.zrefOverride);
|
||||||
|
const initialStepOverride = this.parseOptionalNumber(body.initialStepOverride);
|
||||||
|
|
||||||
|
return this.runs.createRun({
|
||||||
|
tmpFilePath: file.path,
|
||||||
|
originalFilename: file.originalname,
|
||||||
|
params: { zrefOverride, initialStepOverride },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
@Get()
|
||||||
|
list() {
|
||||||
|
return this.runs.list();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Get(':id')
|
||||||
|
detail(@Param('id') id: string) {
|
||||||
|
return this.runs.getDetail(id);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Delete(':id')
|
||||||
|
async remove(@Param('id') id: string) {
|
||||||
|
return this.runs.delete(id);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Get(':id/download/bin')
|
||||||
|
async downloadBin(@Param('id') id: string, @Res() res: Response) {
|
||||||
|
const row = this.runs.getRowOrThrow(id);
|
||||||
|
const report = row.report_json ? JSON.parse(row.report_json) : null;
|
||||||
|
if (!report?.mergedBinFile) throw new NotFoundException('Fichier .bin fusionne non disponible pour ce run');
|
||||||
|
const filePath = path.join(this.runs.workDirFor(id), report.mergedBinFile);
|
||||||
|
res.download(filePath, report.mergedBinFile);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Get(':id/download/csv')
|
||||||
|
async downloadCsv(@Param('id') id: string, @Res() res: Response) {
|
||||||
|
const filePath = path.join(this.runs.workDirFor(id), 'report.csv');
|
||||||
|
if (!fs.existsSync(filePath)) throw new NotFoundException('report.csv non disponible pour ce run');
|
||||||
|
res.download(filePath, `report_${id}.csv`);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Get(':id/download/zip')
|
||||||
|
async downloadZip(@Param('id') id: string, @Res() res: Response) {
|
||||||
|
const dir = this.runs.workDirFor(id);
|
||||||
|
if (!fs.existsSync(dir)) throw new NotFoundException('Resultats non disponibles pour ce run');
|
||||||
|
|
||||||
|
res.setHeader('Content-Type', 'application/zip');
|
||||||
|
res.setHeader('Content-Disposition', `attachment; filename="run_${id}.zip"`);
|
||||||
|
|
||||||
|
const archive = archiver('zip', { zlib: { level: 6 } });
|
||||||
|
archive.on('error', (err) => {
|
||||||
|
res.status(500).end(`Erreur zip: ${err.message}`);
|
||||||
|
});
|
||||||
|
archive.pipe(res);
|
||||||
|
archive.directory(dir, false);
|
||||||
|
await archive.finalize();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Get(':id/images/:filename')
|
||||||
|
async image(@Param('id') id: string, @Param('filename') filename: string, @Res() res: Response) {
|
||||||
|
const safeName = path.basename(filename);
|
||||||
|
if (safeName !== filename) throw new BadRequestException('Nom de fichier invalide');
|
||||||
|
const filePath = path.join(this.runs.workDirFor(id), 'images', safeName);
|
||||||
|
try {
|
||||||
|
await fsp.access(filePath);
|
||||||
|
} catch {
|
||||||
|
throw new NotFoundException('Image introuvable');
|
||||||
|
}
|
||||||
|
res.sendFile(filePath);
|
||||||
|
}
|
||||||
|
|
||||||
|
private parseOptionalNumber(v: string | undefined): number | undefined {
|
||||||
|
if (v === undefined || v === null || v === '') return undefined;
|
||||||
|
const n = Number(v);
|
||||||
|
if (!Number.isFinite(n)) throw new BadRequestException(`Valeur numerique invalide: ${v}`);
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
}
|
||||||
12
src/runs/runs.module.ts
Normal file
12
src/runs/runs.module.ts
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
import { Module } from '@nestjs/common';
|
||||||
|
import { PipelineModule } from '../pipeline/pipeline.module';
|
||||||
|
import { DatabaseService } from './database.service';
|
||||||
|
import { RunsController } from './runs.controller';
|
||||||
|
import { RunsService } from './runs.service';
|
||||||
|
|
||||||
|
@Module({
|
||||||
|
imports: [PipelineModule],
|
||||||
|
controllers: [RunsController],
|
||||||
|
providers: [DatabaseService, RunsService],
|
||||||
|
})
|
||||||
|
export class RunsModule {}
|
||||||
167
src/runs/runs.service.ts
Normal file
167
src/runs/runs.service.ts
Normal file
|
|
@ -0,0 +1,167 @@
|
||||||
|
import { Injectable, Logger, NotFoundException } from '@nestjs/common';
|
||||||
|
import * as fs from 'fs/promises';
|
||||||
|
import * as path from 'path';
|
||||||
|
import { v4 as uuidv4 } from 'uuid';
|
||||||
|
import { resultsDir, uploadsDir } from '../config';
|
||||||
|
import { PipelineService } from '../pipeline/pipeline.service';
|
||||||
|
import { RunParamsInput } from '../pipeline/types';
|
||||||
|
import { DatabaseService, RunRow } from './database.service';
|
||||||
|
|
||||||
|
export interface CreateRunOptions {
|
||||||
|
tmpFilePath: string;
|
||||||
|
originalFilename: string;
|
||||||
|
params: RunParamsInput;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface QueueJob {
|
||||||
|
id: string;
|
||||||
|
inputPath: string;
|
||||||
|
originalFilename: string;
|
||||||
|
params: RunParamsInput;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Injectable()
|
||||||
|
export class RunsService {
|
||||||
|
private readonly logger = new Logger(RunsService.name);
|
||||||
|
private queue: QueueJob[] = [];
|
||||||
|
private processing = false;
|
||||||
|
|
||||||
|
constructor(
|
||||||
|
private readonly db: DatabaseService,
|
||||||
|
private readonly pipeline: PipelineService,
|
||||||
|
) {}
|
||||||
|
|
||||||
|
async createRun(opts: CreateRunOptions): Promise<{ id: string }> {
|
||||||
|
const id = uuidv4();
|
||||||
|
const runUploadDir = path.join(uploadsDir(), id);
|
||||||
|
await fs.mkdir(runUploadDir, { recursive: true });
|
||||||
|
|
||||||
|
const ext = this.extensionOf(opts.originalFilename);
|
||||||
|
const destPath = path.join(runUploadDir, `input${ext}`);
|
||||||
|
await fs.rename(opts.tmpFilePath, destPath);
|
||||||
|
|
||||||
|
this.db.insert({
|
||||||
|
id,
|
||||||
|
original_filename: opts.originalFilename,
|
||||||
|
status: 'pending',
|
||||||
|
created_at: new Date().toISOString(),
|
||||||
|
params_json: JSON.stringify(opts.params),
|
||||||
|
});
|
||||||
|
|
||||||
|
this.queue.push({ id, inputPath: destPath, originalFilename: opts.originalFilename, params: opts.params });
|
||||||
|
void this.processQueue();
|
||||||
|
|
||||||
|
return { id };
|
||||||
|
}
|
||||||
|
|
||||||
|
private async processQueue() {
|
||||||
|
if (this.processing) return;
|
||||||
|
this.processing = true;
|
||||||
|
try {
|
||||||
|
while (this.queue.length > 0) {
|
||||||
|
const job = this.queue.shift()!;
|
||||||
|
await this.runJob(job);
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
this.processing = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async runJob(job: QueueJob) {
|
||||||
|
this.db.updateStatus(job.id, 'running');
|
||||||
|
const workDir = path.join(resultsDir(), job.id);
|
||||||
|
await fs.mkdir(workDir, { recursive: true });
|
||||||
|
|
||||||
|
const onProgress = (msg: string) => {
|
||||||
|
this.logger.log(`[${job.id}] ${msg}`);
|
||||||
|
this.db.appendLog(job.id, msg);
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
const report = await this.pipeline.execute(job.id, job.inputPath, job.originalFilename, workDir, job.params, onProgress);
|
||||||
|
this.db.complete(job.id, JSON.stringify(report));
|
||||||
|
onProgress('Run termine avec succes.');
|
||||||
|
} catch (err: any) {
|
||||||
|
this.logger.error(`Run ${job.id} en erreur: ${err.message}`);
|
||||||
|
this.db.appendLog(job.id, `ERREUR FATALE: ${err.message}`);
|
||||||
|
this.db.fail(job.id, err.message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
list() {
|
||||||
|
return this.db.list().map((r) => this.toSummary(r));
|
||||||
|
}
|
||||||
|
|
||||||
|
getDetail(id: string) {
|
||||||
|
const row = this.db.get(id);
|
||||||
|
if (!row) throw new NotFoundException(`Run ${id} introuvable`);
|
||||||
|
return {
|
||||||
|
id: row.id,
|
||||||
|
originalFilename: row.original_filename,
|
||||||
|
status: row.status,
|
||||||
|
createdAt: row.created_at,
|
||||||
|
finishedAt: row.finished_at,
|
||||||
|
params: row.params_json ? JSON.parse(row.params_json) : null,
|
||||||
|
report: row.report_json ? JSON.parse(row.report_json) : null,
|
||||||
|
log: row.log,
|
||||||
|
error: row.error,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
getRowOrThrow(id: string): RunRow {
|
||||||
|
const row = this.db.get(id);
|
||||||
|
if (!row) throw new NotFoundException(`Run ${id} introuvable`);
|
||||||
|
return row;
|
||||||
|
}
|
||||||
|
|
||||||
|
workDirFor(id: string): string {
|
||||||
|
return path.join(resultsDir(), id);
|
||||||
|
}
|
||||||
|
|
||||||
|
async delete(id: string) {
|
||||||
|
this.getRowOrThrow(id); // leve NotFoundException si le run n'existe pas
|
||||||
|
this.db.remove(id);
|
||||||
|
const work = this.workDirFor(id);
|
||||||
|
const upload = path.join(uploadsDir(), id);
|
||||||
|
await fs.rm(work, { recursive: true, force: true });
|
||||||
|
await fs.rm(upload, { recursive: true, force: true });
|
||||||
|
return { id };
|
||||||
|
}
|
||||||
|
|
||||||
|
private toSummary(row: RunRow) {
|
||||||
|
let summary: any = null;
|
||||||
|
if (row.report_json) {
|
||||||
|
try {
|
||||||
|
const report = JSON.parse(row.report_json);
|
||||||
|
const okLevels = (report.levels ?? []).filter((l: any) => l.status === 'ok');
|
||||||
|
const first = okLevels[0];
|
||||||
|
const last = okLevels[okLevels.length - 1];
|
||||||
|
summary = {
|
||||||
|
levelsOk: okLevels.length,
|
||||||
|
levelsTotal: report.levels?.length ?? 0,
|
||||||
|
volumeLevel0: first?.volume ?? null,
|
||||||
|
volumeLastLevel: last?.volume ?? null,
|
||||||
|
volumeDeltaPct: first && last && first.volume ? (100 * (last.volume - first.volume)) / first.volume : null,
|
||||||
|
};
|
||||||
|
} catch {
|
||||||
|
summary = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
id: row.id,
|
||||||
|
originalFilename: row.original_filename,
|
||||||
|
status: row.status,
|
||||||
|
createdAt: row.created_at,
|
||||||
|
finishedAt: row.finished_at,
|
||||||
|
error: row.error,
|
||||||
|
summary,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private extensionOf(filename: string): string {
|
||||||
|
const lower = filename.toLowerCase();
|
||||||
|
if (lower.endsWith('.copc.laz')) return '.copc.laz';
|
||||||
|
const idx = lower.lastIndexOf('.');
|
||||||
|
return idx >= 0 ? filename.slice(idx) : '';
|
||||||
|
}
|
||||||
|
}
|
||||||
4
tsconfig.build.json
Normal file
4
tsconfig.build.json
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
{
|
||||||
|
"extends": "./tsconfig.json",
|
||||||
|
"exclude": ["node_modules", "dist", "**/*.spec.ts"]
|
||||||
|
}
|
||||||
23
tsconfig.json
Normal file
23
tsconfig.json
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"module": "commonjs",
|
||||||
|
"declaration": false,
|
||||||
|
"removeComments": true,
|
||||||
|
"emitDecoratorMetadata": true,
|
||||||
|
"experimentalDecorators": true,
|
||||||
|
"allowSyntheticDefaultImports": true,
|
||||||
|
"target": "ES2021",
|
||||||
|
"sourceMap": false,
|
||||||
|
"outDir": "./dist",
|
||||||
|
"baseUrl": "./",
|
||||||
|
"incremental": false,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"strictNullChecks": true,
|
||||||
|
"noImplicitAny": true,
|
||||||
|
"strictBindCallApply": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"noFallthroughCasesInSwitch": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"resolveJsonModule": true
|
||||||
|
}
|
||||||
|
}
|
||||||
Loading…
Reference in a new issue