From be465dfe8653d50eb28c6a49cc4dffa8b63b4210 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Mon, 27 Apr 2026 19:43:57 +0200 Subject: [PATCH 01/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- .../adrs/0004-dynamic-mapping-dsl.md | 123 ++++++++++++++++++ 1 file changed, 123 insertions(+) create mode 100644 docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md new file mode 100644 index 0000000..66cf5b8 --- /dev/null +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -0,0 +1,123 @@ +# 0004 - Dynamic Mapping DSL + +* Status: Accepted +* Deciders: + * maintainers team: Andrés BRAND, Matthieu WALTERSPIELER, Eve BERNHARD, Ferial OUKOUKES, Renny VANDOMBER + * contributors team: `N/A` +* Consulted: Étienne JACQUOT +* Informed: `N/A` +* Date: 2026-04-27 + +## Context and Problem Statement + +Mapping source objects to a target object is a central functionality that enables some IDP features like data connectors, calculated properties and Scorecard definitions among others.That’s why we need to do a prior analysis in order to implement, what we call, a mapping engine. This engine must allow dynamic mapping based on a user given query. This query should be user friendly and standardized. + +Dynamic mapping allows an application to change how it interprets incoming data without a recompilation or redeployment. This is typically achieved by using a Domain Specific Language (DSL) stored in a database or configuration file. Given that the IDP-Core is a Springboot application, in this study we will discuss two main DSL; JSLT and JQ. + +## Decision Drivers + +* Security +* User experience +* Complexity of implementation +* FinOps +* Maintainability +* Modularity +* Readability +* Extensibility +* Scalability + +## Decision Outcome + +Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because JSLT provides a capable DSL that meets our requirements for modularity, maintainability, scalability and extensibility. Its explicit use of helper functions for complex logic, while more verbose than jq, results in clearer, more maintainable scripts and better long-term debugging. + +### Positive Consequences + +* **Modular Logic:** Explicit helper functions enhance script modularity and long-term maintainability. +* **Reusable Transformations:** Logic is easily reused across multiple mappings via defined helpers, supporting extensibility. +* **Open Source Friendly:** The DSL approach allows contributors to extend mapping logic without modifying core Java code. + +### Negative Consequences + +* **Learning Curve:** JSLT is less common than JQ; contributors aund final users may need time to learn its XSLT-inspired syntax. +* **Verbosity:** Simple mappings require more lines of code than equivalent JQ expressions. +* **Complex Use Cases:** Certain scenarios (complex grouping/aggregation, cross-referencing arrays, recursive structures, date/time math, dynamic key generation) remain difficult or unreadable, potentially requiring custom Java functions. + +## Considered Options + +1. **Use the JSLT (JSON Standard Transformation Language) Java library** +2. **Use the Jackson-JQ Java Library** + +## Pros and Cons of the Options + +### 1. Use JSLT (JSON Standard Transformation Language) Java library + +JSLT is a JSON transformation language inspired by XSLT, designed for transforming JSON data using a declarative, functional approach. It supports variables, functions, conditionals, and modular script organization. JSLT uses the pipe operator (`|`) for default values and relies on `if...else` blocks, often inside custom helper functions, for conditional logic. Is used for Data ingestion pipelines, ETL, API payload transformation, and scenarios where mapping logic should be externalized from the core application. + +```text +{ + "identifier": .metadata.namespace + "-" + string(.metadata.id), + "status": if (any([for (.analysis.findings) .severity == "critical"])) + "BLOCK" + else + "ALLOW", + "summary": { + "total_bugs": size([for (.analysis.findings) if (.type == "bug") .]), + "critical_count": size([for (.analysis.findings) if (.severity == "critical") .]) + }, + "tags": join([for (.labels) uppercase(.)], ", "), + "last_seen": .analysis.timestamps.finished_at | .analysis.timestamps.started_at +} +``` + +* Good, because **Performance:** It is a native Java implementation designed for the JVM, performing significantly faster than the Jackson-JQ emulation. (Performance) +* Good, because **Helper Functions:** Requires defining helpers (`def function-name(...)`), which makes the main transformation body cleaner and enhances long-term script modularity and debugging. (Maintainability, Extensibility) +* Good, because of the speed of treament. JSLT java library is at least 5 times faster that the jackson-jq one. (Perfomance) +* Good, because **Extensibility:** Logic is easy to reuse across multiple mappings via defined helpers. (Extensibility) +* Good, because **Standardized Reusability:** By identifying frequently used transformations, we can centralize them into a "Core IDP Library." This ensures that if a transformation rule changes (for example, a naming convention update), we update it in one place instead of modifying hundreds of mapping scripts. (Maintainability / Extensibility) +* Good, because **User-Driven Extensibility:** Allowing users to contribute their own functions to the library empowers the community to solve niche problems while keeping the core engine clean and high-performing. (Contributing in open source mode) +* Good, because **Multilayered Extensibility:** Supports function definition at three levels: Java-native (for performance), Imported Library (for platform standards), and Inline DSL (for user-specific logic). This provides a smooth transition from "No-Code" to "Power-User" without hitting an architectural ceiling. (Extensibility / User experience) +* Good, because **Runtime Extensibility:** JSLT facilitates the injection of custom logic post-deployment via an import mechanism or dynamic script concatenation. This allows users to provide their own "Standard Library" of functions (via Docker volumes or Database records) that the engine can load at runtime, enabling high-tier extensibility in an Open Source, "No-Code" context. (Extensibility / Contributing in open source mode) +* Good, because **Robust Syntax Checking:** JSLT’s Parser.compile() returns detailed metadata (line and column numbers) on failure. This allows us to build a high-quality validation stage that tells the user exactly why and where their custom function is broken, significantly improving the "No-Code" troubleshooting experience. (User experience / Testability) +* Bad, because **Verbosity:** More verbose for complex logic due to the requirement for helper definitions. (Complexity of implementation, Readability) +* Bad, because **Limitations:** Becomes overly complex or unreadable for tasks like complex grouping/aggregation, cross-referencing arrays, recursive structures, date/time math, and dynamic key generation. (Complexity of implementation) +* Bad, because **Maintenance Overhead**: To provide a "no-code" experience for end-users, the core team must build and maintain a custom library of helper functions. This creates a permanent maintenance layer that requires versioning, testing, and documentation. (Complexity of implementation / Maintainability) +* Bad, because **Limited Built-ins:** Some advanced math, date or recursive operations may require custom Java function extensions. (Complexity of implementation) +* Bad, because **Smaller Community:** Has a smaller ecosystem and fewer online resources and answers than JQ. (Contributor and end users UX) +* Bad, because **Higher Entry Barrier:** Even with a library, the initial "blank page" experience for a new user is harder with JSLT than with JQ's simple piping syntax. (Contributor UX) +* Bad, because **Configuration Integrity Risk:** Allowing runtime or volume-mounted functions introduces the risk of "broken" configurations preventing system startup. We must implement a validation stage during the application's lifecycle to ensure all external scripts are syntactically correct before they are utilized. (Complexity of implementation / Reliability) + +### 2. Use the Jackson-JQ Java Library + +JQ is a lightweight, command-line JSON processor with a functional, pipeline-based syntax. It excels at concise, inline transformations and is widely used for scripting and quick data manipulation.Jq uses the alternative operator (`//`) for fallbacks and is designed for highly concise, functional chaining of filters. All logic can be written inline. + +```text +{ + "identifier": "\(.metadata.namespace)-\(.metadata.id | tostring)", + "status": ( + if .analysis.findings | any(.severity == "critical") + then "BLOCK" + else "ALLOW" + end + ), + "summary": { + "total_bugs": [.analysis.findings[] | select(.type == "bug")] | length, + "critical_count": [.analysis.findings[] | select(.severity == "critical")] | length + }, + "tags": (.labels | map(ascii_upcase) | join(", ")), + "last_seen": (.analysis.timestamps.finished_at // .analysis.timestamps.started_at) +} +``` + +* Good, because **Conciseness:** Allows complex logic to be written almost entirely inline, resulting in shorter, dense code blocks. (Readability/Conciseness) +* Good, because **Ubiquity:** JQ is the "de-facto" standard for JSON manipulation; most technical users already know the syntax. (Contributor UX) +* Good, because **Logic:** Provides very flexible and concise string manipulation (for example, `split`, `ascii_downcase`). (Complexity of implementation) +* Bad, because **Readability:** Deeply nested or very complex inline logic can sometimes compromise immediate readability and debuggingg. (Readability, Maintainability) +* Bad, because **Performance Penalty:** The Jackson-JQ library is an emulation of the original C-based JQ and is notably slower on large payloads. (Performance) +* Bad, because **Maintenance Debt:** Lacks formal modularity; code reuse is usually achieved via copy-pasting, which leads to duplication across blueprints. (Maintainability, Extensibility) +* Bad, because **Incomplete Parity:** Jackson-JQ is not a 100% port of the official C-based JQ. It lacks several built-in functions and advanced filters, leading to "documentation drift" where developers write code based on official JQ manuals that fails to execute in the Java environment. (Complexity of implementation / Maintainability) +* Bad, because **Static Extension Model:** Unlike JSLT, which allows for dynamic function definitions within the query text, adding custom logic to Jackson-JQ requires implementing a low-level Java interface and registering it within a Scope. This creates a hard dependency on the Java build lifecycle, meaning new transformation functions cannot be injected at runtime (via a database or volume mount) and instead require a full application recompilation and Docker image redeploy. (Complexity of implementation / Developer Velocity) + +## More Information + +1. [JSLT documentation](https://github.com/schibsted/jslt) +2. [JQ documentation](https://jqlang.org) From 0b6499e4ddff6335bf73fcd3609b590235c9a752 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Tue, 28 Apr 2026 10:48:24 +0200 Subject: [PATCH 02/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- .../styles/config/vocabularies/IDP/accept.txt | 2 ++ .../adrs/0004-dynamic-mapping-dsl.md | 19 +++++++++---------- 2 files changed, 11 insertions(+), 10 deletions(-) diff --git a/.vale/styles/config/vocabularies/IDP/accept.txt b/.vale/styles/config/vocabularies/IDP/accept.txt index 4ba4d3e..a362220 100644 --- a/.vale/styles/config/vocabularies/IDP/accept.txt +++ b/.vale/styles/config/vocabularies/IDP/accept.txt @@ -46,3 +46,5 @@ accessors getters Optionals untracked +Perfomance +Reusability diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index 66cf5b8..67d6e82 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -1,4 +1,4 @@ -# 0004 - Dynamic Mapping DSL +# 0004 - Domain Specific Language (DSL) for Dynamic Mapping implementation * Status: Accepted * Deciders: @@ -10,9 +10,9 @@ ## Context and Problem Statement -Mapping source objects to a target object is a central functionality that enables some IDP features like data connectors, calculated properties and Scorecard definitions among others.That’s why we need to do a prior analysis in order to implement, what we call, a mapping engine. This engine must allow dynamic mapping based on a user given query. This query should be user friendly and standardized. +Mapping source objects to a target object is a central functionality that enables some IDP features like data connectors, calculated properties and Scorecard definitions among others. That’s why we need to do a prior analysis in order to implement, what we call, a mapping engine. This engine must allow dynamic mapping based on a user given query. This query should be user friendly and standardized. -Dynamic mapping allows an application to change how it interprets incoming data without a recompilation or redeployment. This is typically achieved by using a Domain Specific Language (DSL) stored in a database or configuration file. Given that the IDP-Core is a Springboot application, in this study we will discuss two main DSL; JSLT and JQ. +Dynamic mapping allows an application to change how it interprets incoming data without a recompilation or redeployment. This is typically achieved by using a Domain Specific Language (DSL) stored in a database or configuration file. Given that the IDP-Core is a SpringBoot application, in this study we will discuss two main DSL; JSLT and JQ. ## Decision Drivers @@ -28,7 +28,7 @@ Dynamic mapping allows an application to change how it interprets incoming data ## Decision Outcome -Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because JSLT provides a capable DSL that meets our requirements for modularity, maintainability, scalability and extensibility. Its explicit use of helper functions for complex logic, while more verbose than jq, results in clearer, more maintainable scripts and better long-term debugging. +Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because JSLT provides a capable DSL that meets our requirements for modularity, maintainability, scalability and extensibility. Its explicit use of helper functions for complex logic, while more verbose than JQ, results in clearer, more maintainable scripts and better long-term debugging. ### Positive Consequences @@ -38,7 +38,7 @@ Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** bec ### Negative Consequences -* **Learning Curve:** JSLT is less common than JQ; contributors aund final users may need time to learn its XSLT-inspired syntax. +* **Learning Curve:** JSLT is less common than JQ; contributors and final users may need time to learn its XSLT-inspired syntax. * **Verbosity:** Simple mappings require more lines of code than equivalent JQ expressions. * **Complex Use Cases:** Certain scenarios (complex grouping/aggregation, cross-referencing arrays, recursive structures, date/time math, dynamic key generation) remain difficult or unreadable, potentially requiring custom Java functions. @@ -71,7 +71,6 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Good, because **Performance:** It is a native Java implementation designed for the JVM, performing significantly faster than the Jackson-JQ emulation. (Performance) * Good, because **Helper Functions:** Requires defining helpers (`def function-name(...)`), which makes the main transformation body cleaner and enhances long-term script modularity and debugging. (Maintainability, Extensibility) -* Good, because of the speed of treament. JSLT java library is at least 5 times faster that the jackson-jq one. (Perfomance) * Good, because **Extensibility:** Logic is easy to reuse across multiple mappings via defined helpers. (Extensibility) * Good, because **Standardized Reusability:** By identifying frequently used transformations, we can centralize them into a "Core IDP Library." This ensures that if a transformation rule changes (for example, a naming convention update), we update it in one place instead of modifying hundreds of mapping scripts. (Maintainability / Extensibility) * Good, because **User-Driven Extensibility:** Allowing users to contribute their own functions to the library empowers the community to solve niche problems while keeping the core engine clean and high-performing. (Contributing in open source mode) @@ -83,12 +82,12 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Bad, because **Maintenance Overhead**: To provide a "no-code" experience for end-users, the core team must build and maintain a custom library of helper functions. This creates a permanent maintenance layer that requires versioning, testing, and documentation. (Complexity of implementation / Maintainability) * Bad, because **Limited Built-ins:** Some advanced math, date or recursive operations may require custom Java function extensions. (Complexity of implementation) * Bad, because **Smaller Community:** Has a smaller ecosystem and fewer online resources and answers than JQ. (Contributor and end users UX) -* Bad, because **Higher Entry Barrier:** Even with a library, the initial "blank page" experience for a new user is harder with JSLT than with JQ's simple piping syntax. (Contributor UX) +* Bad, because **Higher Entry Barrier:** Even with a library, the initial "blank page" experience for a new user is harder with JSLT than with the JQ simple piping syntax. (Contributor UX) * Bad, because **Configuration Integrity Risk:** Allowing runtime or volume-mounted functions introduces the risk of "broken" configurations preventing system startup. We must implement a validation stage during the application's lifecycle to ensure all external scripts are syntactically correct before they are utilized. (Complexity of implementation / Reliability) ### 2. Use the Jackson-JQ Java Library -JQ is a lightweight, command-line JSON processor with a functional, pipeline-based syntax. It excels at concise, inline transformations and is widely used for scripting and quick data manipulation.Jq uses the alternative operator (`//`) for fallbacks and is designed for highly concise, functional chaining of filters. All logic can be written inline. +JQ is a lightweight, command-line JSON processor with a functional, pipeline-based syntax. It excels at concise, inline transformations and is widely used for scripting and quick data manipulation. Jq uses the alternative operator (`//`) for fallbacks and is designed for highly concise, functional chaining of filters. All logic can be written inline. ```text { @@ -110,8 +109,8 @@ JQ is a lightweight, command-line JSON processor with a functional, pipeline-bas * Good, because **Conciseness:** Allows complex logic to be written almost entirely inline, resulting in shorter, dense code blocks. (Readability/Conciseness) * Good, because **Ubiquity:** JQ is the "de-facto" standard for JSON manipulation; most technical users already know the syntax. (Contributor UX) -* Good, because **Logic:** Provides very flexible and concise string manipulation (for example, `split`, `ascii_downcase`). (Complexity of implementation) -* Bad, because **Readability:** Deeply nested or very complex inline logic can sometimes compromise immediate readability and debuggingg. (Readability, Maintainability) +* Good, because **Logic:** Provides flexible and concise string manipulation (for example, `split`, `ascii_downcase`). (Complexity of implementation) +* Bad, because **Readability:** Deeply nested or complex inline logic can sometimes compromise immediate readability and debugging. (Readability, Maintainability) * Bad, because **Performance Penalty:** The Jackson-JQ library is an emulation of the original C-based JQ and is notably slower on large payloads. (Performance) * Bad, because **Maintenance Debt:** Lacks formal modularity; code reuse is usually achieved via copy-pasting, which leads to duplication across blueprints. (Maintainability, Extensibility) * Bad, because **Incomplete Parity:** Jackson-JQ is not a 100% port of the official C-based JQ. It lacks several built-in functions and advanced filters, leading to "documentation drift" where developers write code based on official JQ manuals that fails to execute in the Java environment. (Complexity of implementation / Maintainability) From a0fa2ced91ae3419e13d29af84e0e5850b345616 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Tue, 28 Apr 2026 10:53:18 +0200 Subject: [PATCH 03/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index 67d6e82..fd7948f 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -53,7 +53,7 @@ Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** bec JSLT is a JSON transformation language inspired by XSLT, designed for transforming JSON data using a declarative, functional approach. It supports variables, functions, conditionals, and modular script organization. JSLT uses the pipe operator (`|`) for default values and relies on `if...else` blocks, often inside custom helper functions, for conditional logic. Is used for Data ingestion pipelines, ETL, API payload transformation, and scenarios where mapping logic should be externalized from the core application. -```text +```json { "identifier": .metadata.namespace + "-" + string(.metadata.id), "status": if (any([for (.analysis.findings) .severity == "critical"])) @@ -87,9 +87,9 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi ### 2. Use the Jackson-JQ Java Library -JQ is a lightweight, command-line JSON processor with a functional, pipeline-based syntax. It excels at concise, inline transformations and is widely used for scripting and quick data manipulation. Jq uses the alternative operator (`//`) for fallbacks and is designed for highly concise, functional chaining of filters. All logic can be written inline. +JQ is a lightweight, command-line JSON processor with a functional, pipeline-based syntax. It excels at concise, inline transformations and is widely used for scripting and quick data manipulation. JQ uses the alternative operator (`//`) for fallbacks and is designed for highly concise, functional chaining of filters. All logic can be written inline. -```text +```json { "identifier": "\(.metadata.namespace)-\(.metadata.id | tostring)", "status": ( From 36d77f2ace6cee1bff04520027de781c1486e34f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Thu, 30 Apr 2026 10:55:01 +0200 Subject: [PATCH 04/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- .../adrs/0004-dynamic-mapping-dsl.md | 82 +++++++++---------- 1 file changed, 38 insertions(+), 44 deletions(-) diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index fd7948f..74a9205 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -10,9 +10,7 @@ ## Context and Problem Statement -Mapping source objects to a target object is a central functionality that enables some IDP features like data connectors, calculated properties and Scorecard definitions among others. That’s why we need to do a prior analysis in order to implement, what we call, a mapping engine. This engine must allow dynamic mapping based on a user given query. This query should be user friendly and standardized. - -Dynamic mapping allows an application to change how it interprets incoming data without a recompilation or redeployment. This is typically achieved by using a Domain Specific Language (DSL) stored in a database or configuration file. Given that the IDP-Core is a SpringBoot application, in this study we will discuss two main DSL; JSLT and JQ. +Mapping source objects to a target model is a core capability that enables IDP features such as data connectors, calculated properties, and Scorecard definitions. To implement what we call a "mapping engine," we must support dynamic transformations based on user-provided queries. This query language should be user-friendly, standardized, and late-bound. Dynamic mapping allows the application to change how it interprets incoming data without requiring recompilation or redeployment; this is achieved by using a Domain Specific Language (DSL) stored in a database or configuration file. Given that the IDP-Core is a Spring Boot application, this study evaluates two primary DSL options: JSLT and JQ. ## Decision Drivers @@ -25,27 +23,30 @@ Dynamic mapping allows an application to change how it interprets incoming data * Readability * Extensibility * Scalability +* User Experience (UX) ## Decision Outcome -Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because JSLT provides a capable DSL that meets our requirements for modularity, maintainability, scalability and extensibility. Its explicit use of helper functions for complex logic, while more verbose than JQ, results in clearer, more maintainable scripts and better long-term debugging. +Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because it provides a robust, JVM-native DSL that meets our requirements for perfomance, modularity, maintainability, and scalability. Its explicit use of helper functions for complex logic—while occasionally more verbose than JQ—results in clearer, more maintainable scripts and superior long-term debuggability. ### Positive Consequences -* **Modular Logic:** Explicit helper functions enhance script modularity and long-term maintainability. -* **Reusable Transformations:** Logic is easily reused across multiple mappings via defined helpers, supporting extensibility. -* **Open Source Friendly:** The DSL approach allows contributors to extend mapping logic without modifying core Java code. +* **Modular Logic:** Explicit helper functions enhance script modularity and simplify long-term maintenance. +* **Reusable Transformations:** Logic is easily shared across multiple mappings via defined helpers, supporting high extensibility. +* **Open Source Friendly:** The DSL approach allows community contributors to extend mapping logic without modifying the core Java codebase. ### Negative Consequences -* **Learning Curve:** JSLT is less common than JQ; contributors and final users may need time to learn its XSLT-inspired syntax. -* **Verbosity:** Simple mappings require more lines of code than equivalent JQ expressions. -* **Complex Use Cases:** Certain scenarios (complex grouping/aggregation, cross-referencing arrays, recursive structures, date/time math, dynamic key generation) remain difficult or unreadable, potentially requiring custom Java functions. +* **Learning Curve:** JSLT is less common than JQ; contributors and users may require time to adapt to its XSLT-inspired syntax. +* **Verbosity:** Simple mappings may require more lines of code than equivalent JQ expressions. +* **Complex Use Cases:** Certain scenarios (for example, complex recursive structures or advanced dynamic key generation) can become difficult to read, potentially necessitating custom function extensions. ## Considered Options 1. **Use the JSLT (JSON Standard Transformation Language) Java library** -2. **Use the Jackson-JQ Java Library** +2. **Use JQ as the mapping DSL** + - 2.a **Use the Jackson-jq (Java library)** + - 2.b **Use the JQ binary (via process execution)** ## Pros and Cons of the Options @@ -70,13 +71,12 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi ``` * Good, because **Performance:** It is a native Java implementation designed for the JVM, performing significantly faster than the Jackson-JQ emulation. (Performance) +* Good, because **Robust Syntax Checking:** JSLT’s Parser.compile() returns detailed metadata (line and column numbers) on failure. This allows us to build a high-quality validation stage that tells the user exactly why and where their custom function is broken, significantly improving the "No-Code" troubleshooting experience. (User experience / Testability) * Good, because **Helper Functions:** Requires defining helpers (`def function-name(...)`), which makes the main transformation body cleaner and enhances long-term script modularity and debugging. (Maintainability, Extensibility) -* Good, because **Extensibility:** Logic is easy to reuse across multiple mappings via defined helpers. (Extensibility) +* Good, because **Multilayered Extensibility:** Supports function definition at three levels: Java-native (for performance), Imported Library (for platform standards), and Inline DSL (for user-specific logic). This provides a smooth transition from "No-Code" to "Power-User" without hitting an architectural ceiling. (Extensibility / User experience) * Good, because **Standardized Reusability:** By identifying frequently used transformations, we can centralize them into a "Core IDP Library." This ensures that if a transformation rule changes (for example, a naming convention update), we update it in one place instead of modifying hundreds of mapping scripts. (Maintainability / Extensibility) * Good, because **User-Driven Extensibility:** Allowing users to contribute their own functions to the library empowers the community to solve niche problems while keeping the core engine clean and high-performing. (Contributing in open source mode) -* Good, because **Multilayered Extensibility:** Supports function definition at three levels: Java-native (for performance), Imported Library (for platform standards), and Inline DSL (for user-specific logic). This provides a smooth transition from "No-Code" to "Power-User" without hitting an architectural ceiling. (Extensibility / User experience) * Good, because **Runtime Extensibility:** JSLT facilitates the injection of custom logic post-deployment via an import mechanism or dynamic script concatenation. This allows users to provide their own "Standard Library" of functions (via Docker volumes or Database records) that the engine can load at runtime, enabling high-tier extensibility in an Open Source, "No-Code" context. (Extensibility / Contributing in open source mode) -* Good, because **Robust Syntax Checking:** JSLT’s Parser.compile() returns detailed metadata (line and column numbers) on failure. This allows us to build a high-quality validation stage that tells the user exactly why and where their custom function is broken, significantly improving the "No-Code" troubleshooting experience. (User experience / Testability) * Bad, because **Verbosity:** More verbose for complex logic due to the requirement for helper definitions. (Complexity of implementation, Readability) * Bad, because **Limitations:** Becomes overly complex or unreadable for tasks like complex grouping/aggregation, cross-referencing arrays, recursive structures, date/time math, and dynamic key generation. (Complexity of implementation) * Bad, because **Maintenance Overhead**: To provide a "no-code" experience for end-users, the core team must build and maintain a custom library of helper functions. This creates a permanent maintenance layer that requires versioning, testing, and documentation. (Complexity of implementation / Maintainability) @@ -85,38 +85,32 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Bad, because **Higher Entry Barrier:** Even with a library, the initial "blank page" experience for a new user is harder with JSLT than with the JQ simple piping syntax. (Contributor UX) * Bad, because **Configuration Integrity Risk:** Allowing runtime or volume-mounted functions introduces the risk of "broken" configurations preventing system startup. We must implement a validation stage during the application's lifecycle to ensure all external scripts are syntactically correct before they are utilized. (Complexity of implementation / Reliability) -### 2. Use the Jackson-JQ Java Library +### 2. Use JQ as the mapping DSL -JQ is a lightweight, command-line JSON processor with a functional, pipeline-based syntax. It excels at concise, inline transformations and is widely used for scripting and quick data manipulation. JQ uses the alternative operator (`//`) for fallbacks and is designed for highly concise, functional chaining of filters. All logic can be written inline. +#### Common Good and Bad of JQ (applies to both sub-options) -```json -{ - "identifier": "\(.metadata.namespace)-\(.metadata.id | tostring)", - "status": ( - if .analysis.findings | any(.severity == "critical") - then "BLOCK" - else "ALLOW" - end - ), - "summary": { - "total_bugs": [.analysis.findings[] | select(.type == "bug")] | length, - "critical_count": [.analysis.findings[] | select(.severity == "critical")] | length - }, - "tags": (.labels | map(ascii_upcase) | join(", ")), - "last_seen": (.analysis.timestamps.finished_at // .analysis.timestamps.started_at) -} -``` +* Good, because **Conciseness:** Highly expressive and concise, allowing complex transformations in fewer lines. (Readability/Conciseness) +* Good, because **Popularity:** Widely known and used in the open source and DevOps communities, lowering the learning curve for contributors. (Adoption) +* Good, because **Powerful Built-ins:** Provides a rich set of built-in functions for string manipulation, filtering, and aggregation. (Functionality) +* Bad, because **Extensibility:** JQ lacks formal modularity; code reuse is achieved by copy-pasting or composing filters, which can lead to duplication. (Maintainability) +* Bad, because **Readability:** Deeply nested or complex inline logic can compromise immediate readability. (Readability) +* Bad, because **Integration:** Integrating JQ into a Java application is less seamless than JSLT, particularly reagrding error handling and type safety. (Integration) + +#### 2.a Use the Jackson-jq Java library -* Good, because **Conciseness:** Allows complex logic to be written almost entirely inline, resulting in shorter, dense code blocks. (Readability/Conciseness) -* Good, because **Ubiquity:** JQ is the "de-facto" standard for JSON manipulation; most technical users already know the syntax. (Contributor UX) -* Good, because **Logic:** Provides flexible and concise string manipulation (for example, `split`, `ascii_downcase`). (Complexity of implementation) -* Bad, because **Readability:** Deeply nested or complex inline logic can sometimes compromise immediate readability and debugging. (Readability, Maintainability) -* Bad, because **Performance Penalty:** The Jackson-JQ library is an emulation of the original C-based JQ and is notably slower on large payloads. (Performance) -* Bad, because **Maintenance Debt:** Lacks formal modularity; code reuse is usually achieved via copy-pasting, which leads to duplication across blueprints. (Maintainability, Extensibility) -* Bad, because **Incomplete Parity:** Jackson-JQ is not a 100% port of the official C-based JQ. It lacks several built-in functions and advanced filters, leading to "documentation drift" where developers write code based on official JQ manuals that fails to execute in the Java environment. (Complexity of implementation / Maintainability) -* Bad, because **Static Extension Model:** Unlike JSLT, which allows for dynamic function definitions within the query text, adding custom logic to Jackson-JQ requires implementing a low-level Java interface and registering it within a Scope. This creates a hard dependency on the Java build lifecycle, meaning new transformation functions cannot be injected at runtime (via a database or volume mount) and instead require a full application recompilation and Docker image redeploy. (Complexity of implementation / Developer Velocity) +* Good, because **Runs in JVM:** No external process required; can be embedded directly in the Java application. (Integration) +* Good, because **No system dependencies:** Works in any environment where Java runs. (Portability) +* Bad, because **Performance:** Jackson-JQ is an emulation and is significantly slower than native JQ or JSLT. (Performance) +* Bad, because **Feature Gaps:** Does not support the full JQ specification; advanced filters may fail. +* Bad, because **Error Handling:** Error messages and debugging are less clear than with native JQ. (Debuggability) -## More Information +#### 2.b Use the JQ binary via process execution -1. [JSLT documentation](https://github.com/schibsted/jslt) -2. [JQ documentation](https://jqlang.org) +* Good, because **Full JQ Feature Set:** Access to 100% of the JQ specification and latest updates.. (Functionality) +* Good, because **Decoupled Lyfecycle:** Allows the mapping engine version to be updated independently of the Java application. (Maintenance) +* Bad, because **Perfomance** While native JQ is optimized in C, its performance is negated in a JVM context by Inter-Process Communication (IPC) overhead and the Double-Serialization Tax (marshalling JSON to/from a system pipe). (Performance) +* Bad, because **System Dependency:** Requires handling a JQ binary complicating deployment and portability. (Portability, Maintenance) +* Bad, because **Process Overhead:** Involves spawning external processes, which adds overhead and complexity to error handling and resource management. (Performance) +* Bad, because **Security:** Running external binaries can introduce security risks if not properly sandboxed. (Security) +* Bad, because **Cold Start Latency:** Every execution requires the OS to fork a process and initialize the binary, creating a performance bottleneck. (Performance) +* Bad, because **Process Management:** If the JVM crashes or a Virtual Thread is interrupted while the JQ process is running, there is a risk creating "Zombie Processes". (Integration) From 1c1db182783127fc3517329b961bf7b163f7d0b7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Thu, 30 Apr 2026 11:19:02 +0200 Subject: [PATCH 05/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index 74a9205..a66784a 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -45,8 +45,8 @@ Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** bec 1. **Use the JSLT (JSON Standard Transformation Language) Java library** 2. **Use JQ as the mapping DSL** - - 2.a **Use the Jackson-jq (Java library)** - - 2.b **Use the JQ binary (via process execution)** + * 2.a **Use the Jackson-jq (Java library)** + * 2.b **Use the JQ binary (via process execution)** ## Pros and Cons of the Options @@ -94,7 +94,7 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Good, because **Powerful Built-ins:** Provides a rich set of built-in functions for string manipulation, filtering, and aggregation. (Functionality) * Bad, because **Extensibility:** JQ lacks formal modularity; code reuse is achieved by copy-pasting or composing filters, which can lead to duplication. (Maintainability) * Bad, because **Readability:** Deeply nested or complex inline logic can compromise immediate readability. (Readability) -* Bad, because **Integration:** Integrating JQ into a Java application is less seamless than JSLT, particularly reagrding error handling and type safety. (Integration) +* Bad, because **Integration:** Integrating JQ into a Java application is less seamless than JSLT, particularly regarding error handling and type safety. (Integration) #### 2.a Use the Jackson-jq Java library From dc9538df30b44c9a01da086fa5c9bf8990899ca4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Thu, 30 Apr 2026 11:31:57 +0200 Subject: [PATCH 06/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- .vale/styles/config/vocabularies/IDP/accept.txt | 2 ++ docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md | 6 +++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/.vale/styles/config/vocabularies/IDP/accept.txt b/.vale/styles/config/vocabularies/IDP/accept.txt index a362220..cda46a8 100644 --- a/.vale/styles/config/vocabularies/IDP/accept.txt +++ b/.vale/styles/config/vocabularies/IDP/accept.txt @@ -48,3 +48,5 @@ Optionals untracked Perfomance Reusability +Lyfecycle +sandboxed \ No newline at end of file diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index a66784a..b871e8f 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -27,7 +27,7 @@ Mapping source objects to a target model is a core capability that enables IDP f ## Decision Outcome -Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because it provides a robust, JVM-native DSL that meets our requirements for perfomance, modularity, maintainability, and scalability. Its explicit use of helper functions for complex logic—while occasionally more verbose than JQ—results in clearer, more maintainable scripts and superior long-term debuggability. +Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because it provides a robust, JVM-native DSL that meets our requirements for perfomance, modularity, maintainability, and scalability. Its explicit use of helper functions for complex logic, while occasionally more verbose than JQ, results in clearer, more maintainable scripts that are easier to troubleshoot over the long term. ### Positive Consequences @@ -102,7 +102,7 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Good, because **No system dependencies:** Works in any environment where Java runs. (Portability) * Bad, because **Performance:** Jackson-JQ is an emulation and is significantly slower than native JQ or JSLT. (Performance) * Bad, because **Feature Gaps:** Does not support the full JQ specification; advanced filters may fail. -* Bad, because **Error Handling:** Error messages and debugging are less clear than with native JQ. (Debuggability) +* Bad, because **Error Handling:** Error messages and debugging are less clear than with native JQ. (Maintainability) #### 2.b Use the JQ binary via process execution @@ -113,4 +113,4 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Bad, because **Process Overhead:** Involves spawning external processes, which adds overhead and complexity to error handling and resource management. (Performance) * Bad, because **Security:** Running external binaries can introduce security risks if not properly sandboxed. (Security) * Bad, because **Cold Start Latency:** Every execution requires the OS to fork a process and initialize the binary, creating a performance bottleneck. (Performance) -* Bad, because **Process Management:** If the JVM crashes or a Virtual Thread is interrupted while the JQ process is running, there is a risk creating "Zombie Processes". (Integration) +* Bad, because **Process Management:** If the JVM crashes or a Virtual Thread is interrupted while the JQ process is running, there is a risk creating "Zombie Processes." (Integration) From ece692313ede4e4986b61555c8fdcd4660b5354e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Thu, 30 Apr 2026 11:34:05 +0200 Subject: [PATCH 07/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- .vale/styles/config/vocabularies/IDP/accept.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.vale/styles/config/vocabularies/IDP/accept.txt b/.vale/styles/config/vocabularies/IDP/accept.txt index cda46a8..d98275a 100644 --- a/.vale/styles/config/vocabularies/IDP/accept.txt +++ b/.vale/styles/config/vocabularies/IDP/accept.txt @@ -49,4 +49,4 @@ untracked Perfomance Reusability Lyfecycle -sandboxed \ No newline at end of file +sandboxed From d9592c241a24c845775610c960be013d976b42e2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Thu, 30 Apr 2026 11:51:55 +0200 Subject: [PATCH 08/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index b871e8f..ae5a7f9 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -114,3 +114,8 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Bad, because **Security:** Running external binaries can introduce security risks if not properly sandboxed. (Security) * Bad, because **Cold Start Latency:** Every execution requires the OS to fork a process and initialize the binary, creating a performance bottleneck. (Performance) * Bad, because **Process Management:** If the JVM crashes or a Virtual Thread is interrupted while the JQ process is running, there is a risk creating "Zombie Processes." (Integration) + +## More Information + +1. [JSLT documentation](https://github.com/schibsted/jslt) +2. [JQ documentation](https://jqlang.org) From 1b2d7be9433519817bc6c1c04916d2da8445d6fa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Thu, 30 Apr 2026 14:10:26 +0200 Subject: [PATCH 09/13] chore(): add adr for the mapping DSL (Domain Specific Language) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Etienne JACQUOT <2057151+etiennej70@users.noreply.github.com> Signed-off-by: Andrés Brand --- .vale/styles/config/vocabularies/IDP/accept.txt | 2 -- .../adrs/0004-dynamic-mapping-dsl.md | 17 ++++++++--------- 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/.vale/styles/config/vocabularies/IDP/accept.txt b/.vale/styles/config/vocabularies/IDP/accept.txt index d98275a..c68813b 100644 --- a/.vale/styles/config/vocabularies/IDP/accept.txt +++ b/.vale/styles/config/vocabularies/IDP/accept.txt @@ -46,7 +46,5 @@ accessors getters Optionals untracked -Perfomance Reusability -Lyfecycle sandboxed diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index ae5a7f9..95f60ca 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -23,11 +23,10 @@ Mapping source objects to a target model is a core capability that enables IDP f * Readability * Extensibility * Scalability -* User Experience (UX) ## Decision Outcome -Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because it provides a robust, JVM-native DSL that meets our requirements for perfomance, modularity, maintainability, and scalability. Its explicit use of helper functions for complex logic, while occasionally more verbose than JQ, results in clearer, more maintainable scripts that are easier to troubleshoot over the long term. +Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because it provides a robust, JVM-native DSL that meets our requirements for performance, modularity, maintainability, and scalability. Its explicit use of helper functions for complex logic, while occasionally more verbose than JQ, results in clearer, more maintainable scripts that are easier to troubleshoot over the long term. ### Positive Consequences @@ -52,9 +51,9 @@ Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** bec ### 1. Use JSLT (JSON Standard Transformation Language) Java library -JSLT is a JSON transformation language inspired by XSLT, designed for transforming JSON data using a declarative, functional approach. It supports variables, functions, conditionals, and modular script organization. JSLT uses the pipe operator (`|`) for default values and relies on `if...else` blocks, often inside custom helper functions, for conditional logic. Is used for Data ingestion pipelines, ETL, API payload transformation, and scenarios where mapping logic should be externalized from the core application. +JSLT is a JSON transformation language inspired by XSLT, designed for transforming JSON data using a declarative, functional approach. It supports variables, functions, conditionals, and modular script organization. JSLT uses the pipe operator (`|`) for default values and relies on `if...else` blocks, often inside custom helper functions, for conditional logic. It is used for data ingestion pipelines, ETL, API payload transformation, and scenarios where mapping logic should be externalized from the core application. -```json +```jslt { "identifier": .metadata.namespace + "-" + string(.metadata.id), "status": if (any([for (.analysis.findings) .severity == "critical"])) @@ -81,7 +80,7 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Bad, because **Limitations:** Becomes overly complex or unreadable for tasks like complex grouping/aggregation, cross-referencing arrays, recursive structures, date/time math, and dynamic key generation. (Complexity of implementation) * Bad, because **Maintenance Overhead**: To provide a "no-code" experience for end-users, the core team must build and maintain a custom library of helper functions. This creates a permanent maintenance layer that requires versioning, testing, and documentation. (Complexity of implementation / Maintainability) * Bad, because **Limited Built-ins:** Some advanced math, date or recursive operations may require custom Java function extensions. (Complexity of implementation) -* Bad, because **Smaller Community:** Has a smaller ecosystem and fewer online resources and answers than JQ. (Contributor and end users UX) +* Bad, because **Smaller Community:** Has a smaller ecosystem and fewer online resources and answers than JQ. (Contributor / end users UX) * Bad, because **Higher Entry Barrier:** Even with a library, the initial "blank page" experience for a new user is harder with JSLT than with the JQ simple piping syntax. (Contributor UX) * Bad, because **Configuration Integrity Risk:** Allowing runtime or volume-mounted functions introduces the risk of "broken" configurations preventing system startup. We must implement a validation stage during the application's lifecycle to ensure all external scripts are syntactically correct before they are utilized. (Complexity of implementation / Reliability) @@ -106,12 +105,12 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi #### 2.b Use the JQ binary via process execution -* Good, because **Full JQ Feature Set:** Access to 100% of the JQ specification and latest updates.. (Functionality) -* Good, because **Decoupled Lyfecycle:** Allows the mapping engine version to be updated independently of the Java application. (Maintenance) -* Bad, because **Perfomance** While native JQ is optimized in C, its performance is negated in a JVM context by Inter-Process Communication (IPC) overhead and the Double-Serialization Tax (marshalling JSON to/from a system pipe). (Performance) +* Good, because **Full JQ Feature Set:** Access to 100% of the JQ specification and latest updates. (Functionality) +* Good, because **Decoupled Lifecycle:** Allows the mapping engine version to be updated independently of the Java application. (Maintenance) +* Bad, because **Performance:** While native JQ is optimized in C, its performance is negated in a JVM context by Inter-Process Communication (IPC) overhead and the Double-Serialization Tax (marshalling JSON to/from a system pipe). (Performance) * Bad, because **System Dependency:** Requires handling a JQ binary complicating deployment and portability. (Portability, Maintenance) * Bad, because **Process Overhead:** Involves spawning external processes, which adds overhead and complexity to error handling and resource management. (Performance) -* Bad, because **Security:** Running external binaries can introduce security risks if not properly sandboxed. (Security) +* Bad, because **Security:** Running external binaries can introduce security risks if not safely run in a mastered OS isoleted context. (Security) * Bad, because **Cold Start Latency:** Every execution requires the OS to fork a process and initialize the binary, creating a performance bottleneck. (Performance) * Bad, because **Process Management:** If the JVM crashes or a Virtual Thread is interrupted while the JQ process is running, there is a risk creating "Zombie Processes." (Integration) From 7c9becf732c9dd8f9565cdbf590f3c3902610846 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Thu, 30 Apr 2026 14:21:55 +0200 Subject: [PATCH 10/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- .vale/styles/config/vocabularies/IDP/accept.txt | 3 +-- docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md | 2 ++ 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.vale/styles/config/vocabularies/IDP/accept.txt b/.vale/styles/config/vocabularies/IDP/accept.txt index c68813b..a567d61 100644 --- a/.vale/styles/config/vocabularies/IDP/accept.txt +++ b/.vale/styles/config/vocabularies/IDP/accept.txt @@ -46,5 +46,4 @@ accessors getters Optionals untracked -Reusability -sandboxed +Performance diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index 95f60ca..a41cc40 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -28,6 +28,8 @@ Mapping source objects to a target model is a core capability that enables IDP f Chosen option: option 1, **"JSLT (JSON Standard Transformation Language),"** because it provides a robust, JVM-native DSL that meets our requirements for performance, modularity, maintainability, and scalability. Its explicit use of helper functions for complex logic, while occasionally more verbose than JQ, results in clearer, more maintainable scripts that are easier to troubleshoot over the long term. +The first implementation will only handle JSLT classic usage. Extensibility through custom functions will come later on to ensure long term security and robustness. + ### Positive Consequences * **Modular Logic:** Explicit helper functions enhance script modularity and simplify long-term maintenance. From 84e2966a0c6395a8323abb850dd017891cdf0e9e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Thu, 30 Apr 2026 14:29:07 +0200 Subject: [PATCH 11/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index a41cc40..60617e9 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -75,7 +75,7 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Good, because **Robust Syntax Checking:** JSLT’s Parser.compile() returns detailed metadata (line and column numbers) on failure. This allows us to build a high-quality validation stage that tells the user exactly why and where their custom function is broken, significantly improving the "No-Code" troubleshooting experience. (User experience / Testability) * Good, because **Helper Functions:** Requires defining helpers (`def function-name(...)`), which makes the main transformation body cleaner and enhances long-term script modularity and debugging. (Maintainability, Extensibility) * Good, because **Multilayered Extensibility:** Supports function definition at three levels: Java-native (for performance), Imported Library (for platform standards), and Inline DSL (for user-specific logic). This provides a smooth transition from "No-Code" to "Power-User" without hitting an architectural ceiling. (Extensibility / User experience) -* Good, because **Standardized Reusability:** By identifying frequently used transformations, we can centralize them into a "Core IDP Library." This ensures that if a transformation rule changes (for example, a naming convention update), we update it in one place instead of modifying hundreds of mapping scripts. (Maintainability / Extensibility) +Good, because **Unified Transformation Standards:** Centralizing common logic into a "Core IDP Library" ensures consistency. When a transformation rule changes (e.g., a naming convention update), it is updated in one place, eliminating the need to modify hundreds of individual mapping scripts.(Maintainability / Extensibility) * Good, because **User-Driven Extensibility:** Allowing users to contribute their own functions to the library empowers the community to solve niche problems while keeping the core engine clean and high-performing. (Contributing in open source mode) * Good, because **Runtime Extensibility:** JSLT facilitates the injection of custom logic post-deployment via an import mechanism or dynamic script concatenation. This allows users to provide their own "Standard Library" of functions (via Docker volumes or Database records) that the engine can load at runtime, enabling high-tier extensibility in an Open Source, "No-Code" context. (Extensibility / Contributing in open source mode) * Bad, because **Verbosity:** More verbose for complex logic due to the requirement for helper definitions. (Complexity of implementation, Readability) @@ -112,7 +112,7 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Bad, because **Performance:** While native JQ is optimized in C, its performance is negated in a JVM context by Inter-Process Communication (IPC) overhead and the Double-Serialization Tax (marshalling JSON to/from a system pipe). (Performance) * Bad, because **System Dependency:** Requires handling a JQ binary complicating deployment and portability. (Portability, Maintenance) * Bad, because **Process Overhead:** Involves spawning external processes, which adds overhead and complexity to error handling and resource management. (Performance) -* Bad, because **Security:** Running external binaries can introduce security risks if not safely run in a mastered OS isoleted context. (Security) +* Bad, because **Security:** External binaries increase the attack surface and can introduce risks if they are not restricted to a secure, isolated execution context. (Security) * Bad, because **Cold Start Latency:** Every execution requires the OS to fork a process and initialize the binary, creating a performance bottleneck. (Performance) * Bad, because **Process Management:** If the JVM crashes or a Virtual Thread is interrupted while the JQ process is running, there is a risk creating "Zombie Processes." (Integration) From fee2993f39d0da31e18544b0c831443d8fb7e2a1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Thu, 30 Apr 2026 14:33:17 +0200 Subject: [PATCH 12/13] chore(): add adr for the mapping DSL (Domain Specific Language) --- docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index 60617e9..1e369ab 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -75,7 +75,7 @@ JSLT is a JSON transformation language inspired by XSLT, designed for transformi * Good, because **Robust Syntax Checking:** JSLT’s Parser.compile() returns detailed metadata (line and column numbers) on failure. This allows us to build a high-quality validation stage that tells the user exactly why and where their custom function is broken, significantly improving the "No-Code" troubleshooting experience. (User experience / Testability) * Good, because **Helper Functions:** Requires defining helpers (`def function-name(...)`), which makes the main transformation body cleaner and enhances long-term script modularity and debugging. (Maintainability, Extensibility) * Good, because **Multilayered Extensibility:** Supports function definition at three levels: Java-native (for performance), Imported Library (for platform standards), and Inline DSL (for user-specific logic). This provides a smooth transition from "No-Code" to "Power-User" without hitting an architectural ceiling. (Extensibility / User experience) -Good, because **Unified Transformation Standards:** Centralizing common logic into a "Core IDP Library" ensures consistency. When a transformation rule changes (e.g., a naming convention update), it is updated in one place, eliminating the need to modify hundreds of individual mapping scripts.(Maintainability / Extensibility) +Good, because **Unified Transformation Standards:** Centralizing common logic into a "Core IDP Library" ensures consistency. When a transformation rule changes, it is updated in one place, eliminating the need to modify hundreds of individual mapping scripts.(Maintainability / Extensibility) * Good, because **User-Driven Extensibility:** Allowing users to contribute their own functions to the library empowers the community to solve niche problems while keeping the core engine clean and high-performing. (Contributing in open source mode) * Good, because **Runtime Extensibility:** JSLT facilitates the injection of custom logic post-deployment via an import mechanism or dynamic script concatenation. This allows users to provide their own "Standard Library" of functions (via Docker volumes or Database records) that the engine can load at runtime, enabling high-tier extensibility in an Open Source, "No-Code" context. (Extensibility / Contributing in open source mode) * Bad, because **Verbosity:** More verbose for complex logic due to the requirement for helper definitions. (Complexity of implementation, Readability) From d706c8b6773931268096230e3d03d6829fd37dac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9s=20Brand?= Date: Mon, 4 May 2026 15:56:08 +0200 Subject: [PATCH 13/13] Update docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Etienne JACQUOT <2057151+etiennej70@users.noreply.github.com> Signed-off-by: Andrés Brand --- docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md index 1e369ab..205857d 100644 --- a/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md +++ b/docs/src/contributing/adrs/0004-dynamic-mapping-dsl.md @@ -110,7 +110,6 @@ Good, because **Unified Transformation Standards:** Centralizing common logic in * Good, because **Full JQ Feature Set:** Access to 100% of the JQ specification and latest updates. (Functionality) * Good, because **Decoupled Lifecycle:** Allows the mapping engine version to be updated independently of the Java application. (Maintenance) * Bad, because **Performance:** While native JQ is optimized in C, its performance is negated in a JVM context by Inter-Process Communication (IPC) overhead and the Double-Serialization Tax (marshalling JSON to/from a system pipe). (Performance) -* Bad, because **System Dependency:** Requires handling a JQ binary complicating deployment and portability. (Portability, Maintenance) * Bad, because **Process Overhead:** Involves spawning external processes, which adds overhead and complexity to error handling and resource management. (Performance) * Bad, because **Security:** External binaries increase the attack surface and can introduce risks if they are not restricted to a secure, isolated execution context. (Security) * Bad, because **Cold Start Latency:** Every execution requires the OS to fork a process and initialize the binary, creating a performance bottleneck. (Performance)