Skip to content

Conversation

@AlbertSmit
Copy link

@AlbertSmit AlbertSmit commented Feb 3, 2026

Note

This is an experiment/brain-child exploring v2.0 improvements.
Not in a finished state. Welcome to discuss.

Overview

This PR introduces v2.0 of @kaliber/firebase-queue with breaking changes and significant improvements driven by real-world usage analysis.


Why These Changes?

Usage Analysis

Analyzed 7 projects and 18 services:

Finding Impact
17/18 services have no error recovery Transient failures become permanent
1 service manually implements retry (~50 LOC) This should be a library feature
1 service implements dynamic pause/resume Useful for external service availability

Code Quality Issues

The original codebase had several code smells:

  • Inconsistent patterns: Mix of pseudo-classes, factory functions, and module patterns
  • Pseudo-class oddity: Queue used new keyword but was not a real constructor (no this, weird instanceof check)
  • No types: Zero TypeScript or JSDoc type annotations
  • Large functions: process() was 155 lines with 6 nested functions

Feature Gap vs BullMQ

Feature Before After
Retry with backoff ❌ Manual (~50 LOC) ✅ Built-in
Pause/Resume ❌ Manual ✅ Built-in
Concurrency limit ✅ Built-in
Observability ✅ Logging + Metrics
Types ✅ JSDoc + TypeScript

Breaking Changes ⚠️

Factory functions instead of constructors:

-const Queue = require('@kaliber/firebase-queue')
-const queue = new Queue({ ... })

+const { createQueue } = require('@kaliber/firebase-queue')
+const queue = createQueue({ ... })

Rationale: The old new Queue() pattern was a pseudo-class that did not actually use this - it was a factory function pretending to be a constructor. Converting to explicit factory functions makes the code honest and consistent.


New Features ✨

🔴 P0: Retry with Backoff

Previously manually implemented in rabobank-jobs (~50 lines)

options: {
  retry: {
    maxAttempts: 5,
    backoff: 'exponential', // or 'linear', 'fixed', custom fn
    initialDelayMs: 1000,
    maxDelayMs: 3600000,
    retryableErrors: (error) => error.code !== 'PERMANENT'
  }
}

🟡 P1: Pause/Resume

Previously manually implemented in synchronization-service

queue.pause()    // Stop processing new tasks
queue.resume()   // Continue processing
queue.isPaused() // Check state

Observability

options: {
  observability: {
    logger: winston,      // Structured logging
    metrics: prometheus,  // Counters, histograms, gauges
    tracer: opentelemetry // Distributed tracing
  }
}

Lifecycle Hooks

options: {
  lifecycle: {
    onTaskClaimed: (taskId, workerId) => {},
    onTaskCompleted: (taskId, durationMs, result) => {},
    onTaskFailed: (taskId, durationMs, error) => {},
    onTaskRetryScheduled: (taskId, attempt, delayMs, error) => {},
    onQueuePaused: () => {},
    onQueueResumed: () => {}
  }
}

Queue Stats

queue.getStats()
// { numWorkers, busyWorkers, totalProcessed, totalFailed, totalRetried, isPaused }

Code Quality Improvements

  • Consistent factory pattern throughout (no more pseudo-classes)
  • Comprehensive JSDoc type annotations on all functions
  • Extended types.ts with 50+ type definitions
  • Extracted helpers to task_helpers.js (smaller, focused functions)
  • Merged pnpm branch (Firebase emulator tests, pnpm, eslint)

Testing

pnpm test  # Requires Java for Firebase emulator

EECOLOR and others added 10 commits December 23, 2025 13:49
- Add types
- Fix typescript configuration
- Remove coverage tool
- Remove outdated firebase test package (move towards Firebase tools)
- Switch to pnpm
…f strategies, add queue management features including pause/resume and statistics, and enhance worker lifecycle tracking with heartbeats and concurrency
…tured logging and metrics for task lifecycle events.
…er to factory functions and add error reporting to retry scheduler.
@socket-security
Copy link

socket-security bot commented Feb 3, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​kaliber/​eslint-plugin@​1.0.3691007893100
Addedfirebase-admin@​13.6.0971009987100
Addedfirebase-tools@​15.3.19810010099100

View full report

@coderabbitai
Copy link

coderabbitai bot commented Feb 3, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • 🔍 Trigger a full review
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch albert-test

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@AlbertSmit AlbertSmit self-assigned this Feb 3, 2026
@AlbertSmit AlbertSmit changed the title v2.0: Factory Pattern + Observability + JSDoc Types [Experimentje]: Factory Pattern + Observability + JSDoc Types Feb 3, 2026
@AlbertSmit AlbertSmit changed the title [Experimentje]: Factory Pattern + Observability + JSDoc Types [Experiment]: Factory Pattern + Observability + JSDoc Types Feb 3, 2026
@EECOLOR
Copy link
Member

EECOLOR commented Feb 3, 2026

@AlbertSmit Make sure to base any pull request on pnpm branches

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants