[1.18] Add docs for global and per-name workflow concurrency limits

daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-concurrency.md

@@ -1,18 +1,65 @@
 ---
 type: docs
-title: Workflow Execution Concurrency
-linkTitle: Workflow Execution Concurrency
+title: Workflow Concurrency Limits
+linkTitle: Concurrency Limits
 weight: 9000
-description: "Configure concurrency for Dapr Workflows to rate limit workflow and activity executions."
+description: "Configure concurrency limits for Dapr Workflows to control how many workflows and activities run simultaneously."
 ---
 
-You can configure the maximum concurrent workflows and activities that can be executed at any one time with the following configuration.
-These limits are imposed on a _per_ sidecar basis, meaning that if you have 10 replicas of your workflow app, the effective limit is 10 times the configured value.
+Dapr provides concurrency limits for workflows and activities at two levels:
 
-Setting these limits can help prevent resource exhaustion on your Dapr sidecar and application, or to drain down a backlog of workflows if there had been a spike in activity causing resource contention.
-These limits do not distinguish between different workflow or activity definitions, so they apply to all workflows and activities running in the sidecar.
+- **Per-sidecar limits** control how many workflows or activities a single Dapr instance can execute concurrently.
+- **Global limits** control the total across all replicas, enforced by the scheduler.
 
-See the [Dapr Configuration documentation]({{% ref configuration-overview.md %}}) for more information on how to apply configuration to your Dapr applications.
+Both levels can be configured independently and work together. Per-sidecar limits protect individual instances from resource exhaustion. Global limits enforce cluster-wide capacity constraints, for example, to respect rate limits on downstream services.
+
+## Per-sidecar limits
+
+Per-sidecar limits restrict concurrency within a single Dapr sidecar. If you have 10 replicas with a per-sidecar limit of 100, the effective cluster-wide capacity is up to 1000.
+
+```yaml
+apiVersion: dapr.io/v1alpha1
+kind: Configuration
+metadata:
+  name: appconfig
+spec:
+  workflow:
+    maxConcurrentWorkflowInvocations: 100
+    maxConcurrentActivityInvocations: 1000
+```
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `maxConcurrentWorkflowInvocations` | int32 | Max concurrent workflow executions per sidecar. Default: unlimited. |
+| `maxConcurrentActivityInvocations` | int32 | Max concurrent activity executions per sidecar. Default: unlimited. |
+
+These limits do not distinguish between different workflow or activity names. They apply to all workflows and activities running in the sidecar.
+
+## Global limits
+
+Global limits enforce a maximum across **all replicas** of your application. The Dapr scheduler divides the limit among its instances and holds back triggers when the limit is reached, dispatching them as capacity becomes available.
+
+### All workflows or all activities
+
+```yaml
+apiVersion: dapr.io/v1alpha1
+kind: Configuration
+metadata:
+  name: appconfig
+spec:
+  workflow:
+    globalMaxConcurrentWorkflowInvocations: 50
+    globalMaxConcurrentActivityInvocations: 200
+```
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `globalMaxConcurrentWorkflowInvocations` | int32 | Max concurrent workflow executions across all replicas. Default: unlimited. |
+| `globalMaxConcurrentActivityInvocations` | int32 | Max concurrent activity executions across all replicas. Default: unlimited. |
+
+### Per-name limits
+
+You can set concurrency limits for specific workflow or activity names. This is useful when certain workflows or activities call rate-limited external services.
 
 ```yaml
 apiVersion: dapr.io/v1alpha1
@@ -21,18 +68,56 @@ metadata:
   name: appconfig
 spec:
   workflow:
-    maxConcurrentWorkflowInvocations: 100 # Default is infinite
-    maxConcurrentActivityInvocations: 1000 # Default is infinite
+    globalMaxConcurrentActivityInvocations: 200
+    activityConcurrencyLimits:
+      - name: SendEmail
+        maxConcurrent: 5
+      - name: CallPaymentAPI
+        maxConcurrent: 10
+    workflowConcurrencyLimits:
+      - name: OrderProcess
+        maxConcurrent: 20
 ```
 
+| Property | Type | Description |
+|----------|------|-------------|
+| `activityConcurrencyLimits` | array | Per-activity-name concurrency limits. |
+| `workflowConcurrencyLimits` | array | Per-workflow-name concurrency limits. |
+| `activityConcurrencyLimits[].name` | string | Activity name to limit. |
+| `activityConcurrencyLimits[].maxConcurrent` | int32 | Max concurrent executions across all replicas for this activity. |
+| `workflowConcurrencyLimits[].name` | string | Workflow name to limit. |
+| `workflowConcurrencyLimits[].maxConcurrent` | int32 | Max concurrent executions across all replicas for this workflow. |
+
+A trigger must satisfy **all** applicable limits. For example, if `globalMaxConcurrentActivityInvocations` is 200 and `SendEmail` has a per-name limit of 5, then at most 5 `SendEmail` activities can run, and all activities combined cannot exceed 200.
+
+## How the levels interact
+
+| Limit type | Scope | Enforcement point | Effect of scaling replicas |
+|------------|-------|-------------------|---------------------------|
+| Per-sidecar | Single instance | Dapr sidecar | Effective max = limit x replicas |
+| Global (type) | All replicas | Scheduler | Fixed total regardless of replicas |
+| Global (per-name) | All replicas | Scheduler | Fixed total regardless of replicas |
+
+When both per-sidecar and global limits are configured, both apply. The global limit prevents the cluster-wide total from exceeding the configured value, while the per-sidecar limit prevents any single instance from consuming too much local resources.
+
+## How global limits work with multiple scheduler replicas
+
+The scheduler divides global limits evenly among its instances using floor division. With a global limit of 100 and 3 scheduler replicas, each scheduler enforces a local limit of 33, for an effective cluster max of 99. This ensures the configured limit is never exceeded.
+
+## Comparison with other rate limiting options
+
+Dapr provides several ways to control concurrency and rate limiting:
+
+| Approach | What it controls | Granularity | Scope |
+|----------|-----------------|-------------|-------|
+| [Workflow concurrency limits]({{% ref "workflow-concurrency.md" %}}) | Workflow and activity executions | Per-type or per-name | Per-sidecar or global |
+| [`app-max-concurrency`]({{% ref "control-concurrency.md" %}}) | All requests and events to an app | All traffic | Per-sidecar |
+| [Rate limit middleware]({{% ref "middleware-rate-limit.md" %}}) | HTTP requests per second | Per remote IP | Per-sidecar |
+
 ## Related links
 
-- [Try out Dapr Workflows using the quickstart]({{% ref workflow-quickstart.md %}})
+- [Dapr Configuration reference]({{% ref configuration-overview.md %}})
+- [Control concurrency and rate limit applications]({{% ref control-concurrency.md %}})
+- [Rate limit middleware]({{% ref middleware-rate-limit.md %}})
 - [Workflow overview]({{% ref workflow-overview.md %}})
 - [Workflow API reference]({{% ref workflow_api.md %}})
-- Try out the following examples:
-   - [Python](https://github.com/dapr/python-sdk/tree/master/examples/demo_workflow)
-   - [JavaScript](https://github.com/dapr/js-sdk/tree/main/examples/workflow)
-   - [.NET](https://github.com/dapr/dotnet-sdk/tree/master/examples/Workflow)
-   - [Java](https://github.com/dapr/java-sdk/tree/master/examples/src/main/java/io/dapr/examples/workflows)
-   - [Go](https://github.com/dapr/go-sdk/tree/main/examples/workflow/README.md)

daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-features-concepts.md

@@ -17,6 +17,7 @@ For more information on how workflow state is managed, see the [workflow archite
 Dapr Workflows are functions you write that define a series of tasks to be executed in a particular order.
 The Dapr Workflow engine takes care of scheduling and execution of the tasks, including managing failures and retries.
 If the app hosting your workflows is scaled out across multiple machines, the workflow engine load balances the execution of workflows and their tasks across multiple machines.
+You can [configure concurrency limits]({{% ref "workflow-concurrency.md" %}}) to control how many workflows and activities run simultaneously, either per-sidecar or globally across all replicas.
 
 There are several different kinds of tasks that a workflow can schedule, including
 - [Activities]({{% ref "workflow-features-concepts.md#workflow-activities" %}}) for executing custom logic

daprdocs/content/en/operations/configuration/configuration-overview.md

@@ -268,12 +268,16 @@ For more information, see:
 
 #### Workflow
 
-The `workflow` section contains properties for configuring [Workflows]({{% ref "workflow-overview.md" %}}).
+The `workflow` section contains properties for configuring [Workflows]({{% ref "workflow-overview.md" %}}). See [Workflow Concurrency Limits]({{% ref "workflow-concurrency.md" %}}) for detailed guidance on how these settings interact.
 
 | Property | Type   | Description |
 |------------------|--------|-----|
-| `maxConcurrentWorkflowInvocations` | int32 | Maximum number of concurrent workflow executions per Dapr sidecar. Default is infinite. |
-| `maxConcurrentActivityInvocations` | int32 | Maximum number of concurrent activity executions per Dapr sidecar. Default is infinite. |
+| `maxConcurrentWorkflowInvocations` | int32 | Maximum concurrent workflow executions per Dapr sidecar. Default is unlimited. |
+| `maxConcurrentActivityInvocations` | int32 | Maximum concurrent activity executions per Dapr sidecar. Default is unlimited. |
+| `globalMaxConcurrentWorkflowInvocations` | int32 | Maximum concurrent workflow executions across all replicas, enforced by the scheduler. Default is unlimited. |
+| `globalMaxConcurrentActivityInvocations` | int32 | Maximum concurrent activity executions across all replicas, enforced by the scheduler. Default is unlimited. |
+| `workflowConcurrencyLimits` | array | Per-workflow-name concurrency limits across all replicas. Each entry has `name` (string) and `maxConcurrent` (int32). |
+| `activityConcurrencyLimits` | array | Per-activity-name concurrency limits across all replicas. Each entry has `name` (string) and `maxConcurrent` (int32). |
 
 #### Scope secret store access
 

daprdocs/content/en/operations/configuration/control-concurrency.md

@@ -8,16 +8,19 @@ description: "Learn how to control how many requests and events can invoke your
 
 Typically, in distributed computing, you may only want to allow for a given number of requests to execute concurrently. Using Dapr's `app-max-concurrency`, you can control how many requests and events can invoke your application simultaneously.
 
-Default `app-max-concurreny` is set to `-1`, meaning no concurrency limit is enforced.
+Default `app-max-concurrency` is set to `-1`, meaning no concurrency limit is enforced.
 
 ## Different approaches
 
-While this guide focuses on `app-max-concurrency`, you can also limit request rate per second using the **`middleware.http.ratelimit`** middleware. However, it's important to understand the difference between the two approaches:
+Dapr provides several approaches to concurrency and rate limiting. It's important to understand the differences:
 
-- `middleware.http.ratelimit`: Time bound and limits the number of requests per second
-- `app-max-concurrency`: Specifies the max number of concurrent requests (and events) at any point of time. 
+| Approach | What it controls | Scope |
+|----------|-----------------|-------|
+| `app-max-concurrency` | Max concurrent requests and events to an app | Per-sidecar |
+| `middleware.http.ratelimit` | HTTP requests per second by remote IP | Per-sidecar |
+| [Workflow concurrency limits]({{% ref workflow-concurrency.md %}}) | Workflow and activity executions, with per-name granularity | Per-sidecar or global (across all replicas) |
 
-See [Rate limit middleware]({{% ref middleware-rate-limit.md %}}) for more information about that approach.
+This guide focuses on `app-max-concurrency`. See [Rate limit middleware]({{% ref middleware-rate-limit.md %}}) and [Workflow Concurrency Limits]({{% ref workflow-concurrency.md %}}) for the other approaches.
 
 ## Demo
 

daprdocs/content/en/reference/components-reference/supported-middleware/middleware-rate-limit.md

@@ -38,7 +38,9 @@ Once the limit is reached, the requests will fail with HTTP Status code *429: To
 The rate limit is enforced independently in each Dapr sidecar, and not cluster-wide.
 {{% /alert %}}
 
-Alternatively, the [max concurrency setting]({{% ref control-concurrency.md %}}) can be used to rate-limit applications and applies to all traffic, regardless of remote IP, protocol, or path.
+Alternatively:
+- The [max concurrency setting]({{% ref control-concurrency.md %}}) can be used to rate-limit applications and applies to all traffic, regardless of remote IP, protocol, or path.
+- [Workflow concurrency limits]({{% ref workflow-concurrency.md %}}) provide per-workflow and per-activity concurrency control, including global limits enforced across all replicas by the scheduler.
 
 ## Dapr configuration