diff --git a/docs/commands/fs/deploy.md b/docs/commands/fs/deploy.md new file mode 100644 index 00000000..c80ecb91 --- /dev/null +++ b/docs/commands/fs/deploy.md @@ -0,0 +1,212 @@ +# `deploy` Command + +The `deploy` command imports workspace items from a local source into a target Fabric workspace. + +You can deploy: + +- all items from the source workspace +- specific folders that contain items +- specific items + +**Supported Types:** + +- All Fabric workspace item types that support definition import (e.g., `.Notebook`, `.DataPipeline`, `.Report` ) + +**Usage:** + +``` +fab deploy --config [--target_env ] [--params ] [--force] +``` + +**Parameters:** + +- `--config `: Path to the deployment configuration YAML file. **Required**. +- `--target_env, -tenv `: Environment name used to select environment-specific settings from the configuration file and the parameter file (if present). Optional. +- `--params, -P `: JSON-formatted parameters provided to the deployment process at runtime. Optional. +- `--force, -f`: Run the deployment without interactive confirmation prompts. Optional. + +**Example:** + +``` +fab deploy --config config.yml --target_env dev +``` + +--- + +### Deployment Behavior + +The scope of the deployment, its operations (e.g., publish, unpublish), and settings are defined entirely by the configurations set in a YAML file. + +During execution, the command: + +- publishes items found in the source into the target workspace +- resolves dependencies between items automatically when logical IDs are present +- unpublishes items from the target workspace that no longer exist in the source + +By default, **both publish and unpublish operations are enabled and executed**. +To disable either operation for a specific environment, it must be explicitly skipped in the configuration file. + +The deployment to the Fabric workspaces is executed via the Fabric REST APIs. + +--- + +### Configuration Behavior + +- The configuration file controls what is published and unpublished. +- Publish and unpublish operations are enabled by default. +- Skipping publish or unpublish must be explicitly defined per environment. +- Target Environment selection is resolved only when environment mappings are present in the configuration. +- If `--target_env` is not specified, the configuration fields must not contain environment-mappings. + +--- + +### Configuration File + +#### Minimal Configuration (No Environment Mappings) + +```yaml +core: + workspace_id: "12345678-1234-1234-1234-123456789abc" + repository_directory: "." +``` + +#### Configuration Fields + +| Field | Description | Mandatory | +|------|------------|-----------| +| `core.workspace_id` | Target workspace ID (GUID). Takes precedence over `workspace`. | ✅ Yes | +| `core.workspace` | Target workspace name. Alternative to `workspace_id`. | ❌ No | +| `core.repository_directory` | Path to local directory containing item files. | ✅ Yes | +| `core.item_types_in_scope` | List of item types included in deployment. | ❌ No | +| `core.parameter` | Path to parameter file. | ❌ No | +| `publish` | Controls publishing behavior. | ❌ No | +| `unpublish` | Controls unpublishing behavior. | ❌ No | +| `features` | Set feature flags. | ❌ No | +| `constants` | Override constant values. | ❌ No | + +Relative paths for `repository_directory` and `parameter` fields are resolved relative to the configuration file location. + +--- + +### Publish and Unpublish Settings + +#### Publish + +```yaml +publish: + exclude_regex: "^DONT_DEPLOY.*" # Excludes items matching this pattern from publishing + folder_exclude_regex: "^/legacy" # Excludes items under matching folders from publishing (requires feature flags) + folder_path_to_include: # Publishes only the specified items under matching folder (requires feature flags) + - /subfolder_1 + items_to_include: # Publishes only the specified items (requires feature flags) + - "MainNotebook.Notebook" + skip: # Skips publishing per environment + dev: true + test: false + prod: false +``` + +#### Unpublish + +```yaml +unpublish: + exclude_regex: "^DEBUG.*" # Prevents matching items from being removed + items_to_include: # Unpublishes only the specified items (requires feature flags) + - "OldPipeline.DataPipeline" + skip: # Skips unpublishing per environment + dev: true + test: false + prod: false +``` + +> **Note** +> While selective deployment is supported, it is not recommended due to potential issues with dependency management. Use selective deployment options with caution. + +--- + +### Parameter File + +The parameter file enables environment‑specific transformations by applying targeted replacements within deployed content. + +It can be used to: + +- replace workspace and item identifiers +- replace connection strings and URLs +- update embedded values inside item definitions +- parameterize environment-specific configuration values + +The parameter file is optional and is applied only when specified in the configuration. + +When deploying item files that were not created via Git Integration but instead exported (for example, using the Fabric CLI export command), parameterization should be used to resolve dependencies. Otherwise, items will reference the original item. + +#### Example + +```yaml +find_replace: + - find_value: "dev-connection-string" + replace_value: + dev: "dev-connection-string" + test: "test-connection-string" + prod: "prod-connection-string" +``` + +#### Supported `replace_value` Variables + +- `$workspace.$id` — resolves to the target workspace ID +- `$items...$id` — resolves to the deployed item ID + +Parameterization behavior follows the documented model described here: + +https://microsoft.github.io/fabric-cicd/latest/how_to/parameterization + +--- + +### More Examples + +#### Deployment to a Specific Environment + +```bash +fab deploy --config config.yml --target_env prod +``` + +#### Deployment with Runtime Parameters + +```bash +fab deploy --config config.yml --target_env test -P config_override='{"core":{"item_types_in_scope":["Notebook"]}}' +``` + +--- + +### Preparing Source Content + +#### Using Git Integration + +If the workspace is connected to Git, clone the repository locally: + +```bash +git clone .git +``` + +Use the cloned repository as the deployment source. + +#### Using `fab export` + +`fab export` exports one item at a time and does not include logical IDs. + +```bash +fab export ws.Workspace/MyDataPipeline.DataPipeline -o C:\Users\myuser +``` + +--- + +### Additional Notes + +- The deploy command can be applied to Fabric item files created via Git integration or using the `export` command (utilizes Get Item Definition API). +- `fab deploy` does not require items to be created via Git integration prior to deployment; however, using Git integration is strongly recommended for dependency resolution with minimal parameterization. +- Parameterization is optional and can be applied in any deployment scenario. + +--- + +### Usfull links + +- [Fabric cicd lib](https://microsoft.github.io/fabric-cicd/latest/) \ No newline at end of file diff --git a/docs/commands/index.md b/docs/commands/index.md index 6c66ba47..fa1e87fd 100644 --- a/docs/commands/index.md +++ b/docs/commands/index.md @@ -65,6 +65,7 @@ It supports both Unix-style and Windows-style command names for file system oper | [`start`](./fs/start.md) | Start a resource | | [`stop`](./fs/stop.md) | Stop a resource | | [`unassign`](./fs/unassign.md) | Unassign a resource from a workspace | +| [`deploy`](./fs/deploy.md) | Deploy Fabric workspace items from local source content into a target Microsoft Fabric workspace. | #### [Table Management (table)](tables/index.md) Commands for working with tables in lakehouses, including operations like loading data, optimizing tables, and managing schemas. diff --git a/docs/examples/folder_examples.md b/docs/examples/folder_examples.md index f5f25047..2dc3db4b 100644 --- a/docs/examples/folder_examples.md +++ b/docs/examples/folder_examples.md @@ -173,6 +173,29 @@ fab mv ws1.Workspace/source.Folder ws2.Workspace *Result:* `ws2.Workspace/dest.Folder/source.Folder` +### Deploy Fabric workspace folders containing items + +Deploy specific folders from local content using configuration files. This allows selective deployment of folder contents. + +#### Deploy Specific Folders + +Deploy only items within specific folders to target workspace. + +``` +fab deploy --config config.yml -tenv dev -P config_override='{"publish": {"folders_to_include": ["Production", "Core"]}, "features": ["enable_include_folder", "enable_experimental_features"]}' +``` + +#### Deploy with Folder Exclusions + +Deploy workspace content while excluding specific folders. + +``` +fab deploy --config config.yml -tenv prod -P config_override='{"publish": {"folder_exclude_regex": "^temp|^debug"}, "features": ["enable_exclude_folder", "enable_experimental_features"]}' +``` + +!!! note "Configuration Options" + All the configuration options shown above with `config_override` can also be defined directly in the configuration file instead of passing them as command-line overrides. The `config_override` examples are provided for demonstration and dynamic configuration purposes. + --- For managing folders within OneLake storage (e.g., `Lakehouse/Files`), see [OneLake Examples](./onelake_examples.md) \ No newline at end of file diff --git a/docs/examples/item_examples.md b/docs/examples/item_examples.md index 64045402..d157edda 100644 --- a/docs/examples/item_examples.md +++ b/docs/examples/item_examples.md @@ -345,6 +345,41 @@ Import an item definition from a local directory into the workspace. fab import ws1.Workspace/nb1_imported.Notebook -i /tmp/exports/nb1.Notebook ``` +#### Deploy Items from Configuration + +Deploy items from local workspace content to a target workspace using a configuration file. + +``` +fab deploy --config config.yml -tenv dev +``` + +#### Deploy Specific Item Types + +Deploy only specific types of items using item_types_in_scope. + +``` +fab deploy --config config.yml -tenv dev -P config_override='{"core": {"item_types_in_scope": ["Notebook", "DataPipeline"]}}' +``` + +#### Deploy Specific Items + +Deploy only specified items using items_to_include. + +``` +fab deploy --config config.yml -tenv prod -P config_override='{"publish": {"items_to_include": ["MainNotebook.Notebook", "ProductionPipeline.DataPipeline"]}, "features": ["enable_items_to_include", "enable_experimental_features"]}' +``` + +#### Deploy with Item Exclusions + +Deploy items while excluding specific items or patterns. + +``` +fab deploy --config config.yml -tenv dev -P config_override='{"publish": {"exclude_regex": "^(TEMP|DEBUG|TEST).*"}}' +``` + +!!! note "Configuration Options" + All the configuration options shown above with `config_override` can also be defined directly in the configuration file instead of passing them as command-line overrides. The `config_override` examples are provided for demonstration and dynamic configuration purposes. + ### Start/Stop Mirrored Databases #### Start Mirrored Database diff --git a/docs/examples/workspace_examples.md b/docs/examples/workspace_examples.md index 024e29e4..907d8c97 100644 --- a/docs/examples/workspace_examples.md +++ b/docs/examples/workspace_examples.md @@ -260,6 +260,18 @@ fab export ws1.Workspace -o /ws1.Workspace/lh1.Lakehouse/Files fab export ws1.Workspace -o /ws1.Workspace/lh1.Lakehouse/Files -a ``` +### Deploy Workspace Items + +Deploy workspace items from local source to target workspaces using configuration files. + +#### Deploy Full Workspace + +Deploy entire workspace content to a target workspace. + +``` +fab deploy --config config.yml -tenv dev +``` + ### Copy Workspace Items #### Copy Items Between Workspaces diff --git a/mkdocs.yml b/mkdocs.yml index 89e315dd..9a5ea4d1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -81,6 +81,7 @@ nav: - cd: commands/fs/cd.md - config: commands/config/index.md - cp (copy): commands/fs/cp.md + - deploy: commands/fs/deploy.md - desc: commands/desc/index.md - export: commands/fs/export.md - exists: commands/fs/exists.md