Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
173 changes: 154 additions & 19 deletions docs/features/video-compression.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,11 @@ the app uses [@ffmpeg/ffmpeg](https://github.com/ffmpegwasm/ffmpeg.wasm) v0.12.1
│ │ FfmpegService │ │
│ │ ┌────────────────┐ │ │
│ │ │ compressVideo()│ │ │
│ │ └────────────────┘ │ │
│ │ ┌────────────────┐ │ │
│ │ └───────┬────────┘ │ │
│ │ ┌───────▼────────┐ │ │
│ │ │probeMetadata() │ │ <video> metadata read │
│ │ └───────┬────────┘ │ │
│ │ ┌───────▼────────┐ │ │
│ │ │ loadFFmpeg() │ │ lazy WASM load │
│ │ └────────────────┘ │ │
│ └──────────────────────┘ │
Expand Down Expand Up @@ -76,22 +79,51 @@ when a video file is added to any uppy instance, the compression interceptor:

1. checks if the file is a video (by mime type)
2. applies size gating — skips compression for files < 5 MB or > limits
3. lazy-loads the ffmpeg wasm core (only on first video)
4. pauses the uppy upload queue
5. compresses the video with h.264/aac encoding
6. replaces the original file in uppy with the compressed version
7. resumes the upload queue
3. **probes video metadata** via a temporary `<video>` element (width, height, duration) — fast, no wasm needed
4. lazy-loads the ffmpeg wasm core (only on first video — runs after probe)
5. pauses the uppy upload queue
6. compresses the video with h.264/aac encoding, **conditionally applying a scale filter** based on probe results
7. replaces the original file in uppy with the compressed version
8. resumes the upload queue

### compression parameters

| parameter | desktop | mobile | purpose |
|-----------|---------|--------|---------|
| `maxHeight` | 720 | 480 | scale output height (aspect preserved) |
| `maxHeight` | 720 | 480 | target height ceiling for the scale filter (aspect preserved via `-2:h`) |
| `crf` | 28 | 30 | quality factor (0-51, lower=better) |
| `preset` | `fast` | `ultrafast` | encoding speed |
| `audioBitrate` | `128k` | `96k` | aac audio bitrate |
| `movflags` | `+faststart` | `+faststart` | enables streaming playback |

### conditional scale filter

the `-vf scale=-2:${maxHeight}` ffmpeg filter is **only added when necessary**:

| probe result | source height | action |
|--------------|---------------|--------|
| metadata available | height ≤ maxHeight | **skip scale** — source already fits, no re-encode overhead |
| metadata available | height > maxHeight | **apply scale** — downscale to maxHeight, preserve aspect ratio |
| probe failed/timed out | unknown | **apply scale** (safe fallback — preserves previous behaviour) |

this avoids an unnecessary decode/filter/encode pass when uploading a video that is already within the resolution ceiling (e.g. a 480p clip uploaded on mobile where `maxHeight = 480`).

#### `probeMetadata()` implementation

`FfmpegService.probeMetadata(file)` extracts metadata before wasm loads:

```typescript
private probeMetadata(file: File): Promise<VideoMetadata | null>
```

- creates a temporary `<video>` element with `URL.createObjectURL(file)` and `preload=metadata`
- resolves with `{ width, height, durationSec }` on the `loadedmetadata` event
- resolves `null` if dimensions are zero, duration is non-finite, an error fires, or the **5-second timeout** (`PROBE_TIMEOUT_SEC`) elapses
- always calls `URL.revokeObjectURL()` via a `cleanup()` function to prevent memory leaks
- returns `null` for non-video files (browser won't load metadata)

**why before `loadFFmpeg()`?** the probe is a lightweight browser api call (milliseconds) whereas loading the wasm core takes 1-3 seconds. by probing first, the scale decision is ready before wasm finishes loading.

### size gating

| condition | action |
Expand Down Expand Up @@ -159,6 +191,7 @@ the installed `@ffmpeg/core` v0.12.10 is the **single-thread** build. this was a
- lower resolution cap (480p vs 720p)
- lower audio bitrate (96k vs 128k)
- size gate at 200 MB (vs 500 MB desktop)
- **conditional scale filter** — videos already ≤ 480p skip the scale pass entirely, saving significant encode time on low-end devices

### 3. cross-browser compatibility

Expand All @@ -185,36 +218,138 @@ the installed `@ffmpeg/core` v0.12.10 is the **single-thread** build. this was a

### 4. wasm asset loading

assets are loaded lazily from `assets/ffmpeg/`:
assets are loaded lazily from `assets/ffmpeg/` using `document.baseURI` as the base (not `window.location.origin`) to correctly resolve under locale-prefixed paths like `/en-US/`:

```typescript
await this.ffmpeg.load({
coreURL: new URL('assets/ffmpeg/ffmpeg-core.js', window.location.origin).toString(),
wasmURL: new URL('assets/ffmpeg/ffmpeg-core.wasm', window.location.origin).toString(),
classWorkerURL: new URL('assets/ffmpeg/worker.js', window.location.origin).toString(),
coreURL: new URL('assets/ffmpeg/ffmpeg-core.js', document.baseURI).toString(),
wasmURL: new URL('assets/ffmpeg/ffmpeg-core.wasm', document.baseURI).toString(),
classWorkerURL: new URL('assets/ffmpeg/worker.js', document.baseURI).toString(),
});
```

**why `document.baseURI`?** the app is served under `<base href="/en-US/">` on staging/production. `window.location.origin` returns `https://app.p2-stage.practera.com` — missing the locale prefix — so asset requests 404 and the SPA returns `index.html` instead. `document.baseURI` respects the `<base>` tag and resolves to the correct path.

#### vendored worker.js

`projects/v3/src/assets/ffmpeg/worker.js` is a **vendored copy** of `node_modules/@ffmpeg/ffmpeg/dist/esm/worker.js` (v0.12.15). the upstream file has ES module `import` statements referencing `./const.js` and `./errors.js` — but angular's asset copy only copies the files we list in `angular.json`, not the full `dist/esm/` tree. when the worker is loaded as a standalone asset at `/en-US/assets/ffmpeg/worker.js`, those sibling modules don't exist at the serving path, so the worker silently fails and `ffmpeg.load()` hangs forever.

the fix: all dependencies from `const.js` and `errors.js` are inlined directly into `worker.js`. the file header documents the exact upstream version and which constants were inlined. **keep this in sync when upgrading `@ffmpeg/ffmpeg`.**

#### angular.json asset config

```json
{
"glob": "**/*",
"input": "node_modules/@ffmpeg/ffmpeg/dist/esm",
"output": "./assets/ffmpeg"
},
"projects/v3/src/assets/ffmpeg"
```

the node_modules glob copies the upstream files first. then the local `projects/v3/src/assets/ffmpeg` directory is listed **after**, so its files (including the vendored `worker.js`) override the upstream versions. this is intentional — it provides a fallback for any new files added in future `@ffmpeg/ffmpeg` releases while ensuring the vendored worker takes priority.

**asset sizes:**
- `ffmpeg-core.wasm`: ~31 MB (downloaded once, browser-cached)
- `ffmpeg-core.js`: ~200 KB (emscripten glue)
- `worker.js`: ~2 KB

**caching strategy:** the wasm file is loaded once and the `FFmpeg` instance is reused. subsequent compressions skip the load step entirely.
**caching strategy:** the wasm file is loaded once and the `FFmpeg` instance is reused. subsequent compressions skip the load step entirely (unless `terminate()` was called, in which case the next compression will lazy-load again).

### 5. known limitations

1. **memory**: wasm heap limited to 2 GB. very large files (>500 MB) may cause oom on constrained devices.
2. **no hardware acceleration**: wasm cannot access gpu/videotoolbox — encoding is pure cpu.
3. **single-thread**: only uses one cpu core. multi-thread would help but requires coop/coep headers and drops ios support.
4. **asset duplication in angular.json**: both `node_modules/@ffmpeg/ffmpeg/dist/esm` and `projects/v3/src/assets/ffmpeg` are copied to output. the node_modules copy should be removed once local assets are confirmed stable.

---

## transcoding

`FfmpegService.transcodeToMp4(file)` converts any user-supplied video to h.264/aac mp4 without custom compression parameters. this is used when the goal is format normalization (e.g. `.webm` → `.mp4`) rather than size reduction. it uses the same lazy wasm load and exec timeout as `compressVideo()`.

---

## error handling

### exec timeout

all `ffmpeg.exec()` calls have a timeout:
- **desktop**: 10 minutes
- **mobile**: 5 minutes
- configurable via `CompressionOptions.timeout` (ms, -1 = unlimited)

if the timeout fires, ffmpeg throws and the preprocessor `catch` block uploads the original file uncompressed.

### compression cancellation (`cancelCompression`)

`UppyUploaderService.cancelCompression()` terminates the wasm worker mid-encode and resets all state:

```
cancelCompression() → ffmpegService.terminate() → progress$.next(null) → compressingUppy = null
```

called automatically from:
- `FileUploadComponent.ngOnDestroy()` — triggered by route navigation away from assessment
- `UppyUploaderComponent.ngOnDestroy()` — triggered by modal dismissal or route change

after termination, `FfmpegService` creates a fresh `FFmpeg` instance so the next compression starts cleanly.

### navigation guards

| scenario | protection |
|----------|------------|
| user clicks another task / browser back (angular route) | `SinglePageDeactivateGuard` — shows `window.confirm()` dialog, cancels compression if user confirms leave |
| user closes tab or reloads | `beforeunload` event listener on `UppyUploaderService` — browser shows native "leave page?" prompt |
| user swipe-dismisses modal (ionic gesture) | `canDismiss` on modal — returns `false` while `compressingUppy !== null` |
| modal dismissed programmatically (e.g. route forces modal stack clear) | `UppyUploaderComponent.ngOnDestroy()` calls `cancelCompression()` |

### subscription leak prevention

the progress subscription in `registerCompressionPreProcessor` uses `try/finally` to guarantee `sub.unsubscribe()` runs even if `compressVideo()` throws:

```typescript
const sub = ffmpegService.progress$.subscribe(...);
try {
result = await ffmpegService.compressVideo(file);
} finally {
sub.unsubscribe();
}
```

### graceful fallback

if compression fails for any reason (timeout, wasm crash, out of memory), the preprocessor `catch` block:
1. logs the error
2. nulls `compressingUppy`
3. emits `progress: null` to hide the overlay
4. lets uppy proceed with the **original uncompressed file**

the user is never blocked from uploading.

## future improvements

| priority | improvement | effort |
|----------|-------------|--------|
| p1 | remove angular.json asset duplication | low |
| p2 | add feature flag to toggle compression | low |
| p3 | compression quality preview (thumbnail before/after) | medium |
| p4 | evaluate `@ffmpeg/core-mt` when ios safari adds SharedArrayBuffer | medium |
| p5 | webcodecs api as alternative for simple transcode (chrome 94+) | high |
| p1 | add feature flag to toggle compression | low |
| p2 | compression quality preview (thumbnail before/after) | medium |
| p3 | evaluate `@ffmpeg/core-mt` when ios safari adds SharedArrayBuffer | medium |
| p4 | use `durationSec` from probe to apply stricter crf/preset for very short clips | low |

### not planned

| item | reason |
|------|--------|
| webcodecs api as ffmpeg.wasm replacement | not viable — caniuse marks it "limited availability"; ios/safari support is video-only (no audio encode pipeline); firefox android unsupported; no built-in demux api (would require mp4box.js or similar, negating simplicity gains); still a w3c working draft as of apr 2026. revisit when it reaches full cross-browser baseline |

---

## changelog

| date | change |
|------|--------|
| apr 2026 | added `probeMetadata()` + conditional scale filter — skips `-vf scale` when source already fits within `maxHeight` |
| apr 2026 | fixed service duplication bug — removed `UppyUploaderService` from `ComponentsModule.providers`; `providedIn: 'root'` services must not be declared in module providers as this creates a second instance in lazy-loaded contexts, breaking `SinglePageDeactivateGuard` |
| nov 2025 | added `cancelCompression()`, `beforeunload` guard, `CanDeactivate` guard, `try/finally` subscription cleanup, and 10/5-minute exec timeouts |
| nov 2025 | fixed wasm load from `window.location.origin` → `document.baseURI` to support `/en-US/` base href |
| nov 2025 | vendored `worker.js` with inlined imports to fix silent failure when angular asset copy omits sibling modules |
2 changes: 0 additions & 2 deletions projects/v3/src/app/components/components.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ import { ToggleLabelDirective } from '../directives/toggle-label/toggle-label.di
import { TrafficLightGroupComponent } from './traffic-light-group/traffic-light-group.component';
import { UppyUploaderComponent } from './uppy-uploader/uppy-uploader.component';
import { FileUploadComponent } from './file-upload/file-upload.component';
import { UppyUploaderService } from './uppy-uploader/uppy-uploader.service';
import { FilePopupComponent } from './file-popup/file-popup.component';
import { SliderComponent } from './slider/slider.component';
import { LanguageDetectionPipe } from '../pipes/language.pipe';
Expand Down Expand Up @@ -163,6 +162,5 @@ const largeCircleDefaultConfig = {
FileUploadComponent,
LanguageDetectionPipe,
],
providers: [UppyUploaderService]
})
export class ComponentsModule { }
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ export class FileUploadComponent implements OnInit, OnDestroy {

ngOnDestroy(): void {
this.compressionSub?.unsubscribe();
this.uppyUploaderService.cancelCompression();
this.uppy.destroy();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -119,13 +119,9 @@ export class UppyUploaderComponent implements OnInit, OnDestroy {

ngOnDestroy() {
this.compressionSub?.unsubscribe();
this.uppyUploaderService.cancelCompression();
if (this.uppy) {
// eslint-disable-next-line no-console
this.uppy.off("upload-success", (res) => console.info(res));

// eslint-disable-next-line no-console
this.uppy.off("complete", (res) => console.info(res));
this.uppy.resetProgress();
this.uppy.destroy();
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ describe('UppyUploaderService', () => {
ffmpegServiceSpy = jasmine.createSpyObj('FfmpegService', [
'shouldCompress',
'compressVideo',
'terminate',
], {
progress$: new Subject(),
});
Expand Down Expand Up @@ -153,6 +154,33 @@ describe('UppyUploaderService', () => {
});
});

describe('cancelCompression', () => {
it('should do nothing if no compression is active', () => {
service.compressingUppy = null;

service.cancelCompression();

expect(ffmpegServiceSpy.terminate).not.toHaveBeenCalled();
});

it('should terminate ffmpeg and emit null progress when compressing', () => {
const fakeUppy = {} as Uppy<any, any>;
service.compressingUppy = fakeUppy;

const emitted: any[] = [];
const sub = service.compressionProgress$.subscribe(v => emitted.push(v));

service.cancelCompression();

expect(ffmpegServiceSpy.terminate).toHaveBeenCalled();
expect(emitted.length).toBe(1);
expect(emitted[0]).toEqual({ uppy: fakeUppy, progress: null });
expect(service.compressingUppy).toBeNull();

sub.unsubscribe();
});
});

describe('open', () => {
it('should create a modal with backdropDismiss false', async () => {
const mockModal = jasmine.createSpyObj('HTMLIonModalElement', ['present']);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,21 @@ export class UppyUploaderService {
private ffmpegService: FfmpegService,
private ngZone: NgZone,
) {
// warn the user before tab close/reload if compression is active
window.addEventListener('beforeunload', (e) => {
if (this.compressingUppy) {
e.preventDefault();
}
});
}

/** cancel any in-flight compression, terminate the wasm worker, and reset state */
cancelCompression(): void {
if (this.compressingUppy) {
this.ffmpegService.terminate();
this.compressionProgress$.next({ uppy: this.compressingUppy, progress: null });
this.compressingUppy = null;
}
}

/**
Expand Down Expand Up @@ -240,9 +255,13 @@ export class UppyUploaderService {
this.compressionProgress$.next({ uppy, progress: p });
});

const result = await this.ffmpegService.compressVideo(file);
let result;
try {
result = await this.ffmpegService.compressVideo(file);
} finally {
sub.unsubscribe();
}

sub.unsubscribe();
this.ngZone.run(() => {
this.compressionProgress$.next({ uppy, progress: null });
this.compressingUppy = null;
Expand Down
Loading
Loading