diff --git a/.github/skills/interop-generator/SKILL.md b/.github/skills/interop-generator/SKILL.md index 22a4ec56c9..1a918671c1 100644 --- a/.github/skills/interop-generator/SKILL.md +++ b/.github/skills/interop-generator/SKILL.md @@ -1,6 +1,6 @@ --- name: interop-generator -description: Work with, answer questions about, edit, or debug the CsWinRT interop generator (cswinrtinteropgen.exe). Use when the user needs to understand, modify, debug, or extend the interop sidecar generator that produces WinRT.Interop.dll. +description: Use whenever the user mentions the interop generator (cswinrtinteropgen.exe, which produces WinRT.Interop.dll) in any context — debugging, extending, or just understanding how it works. --- # CsWinRT interop generator (`cswinrtinteropgen.exe`) @@ -74,9 +74,15 @@ WinRT.Interop.Generator/ ├── Factories/ # Type/member/attribute creation factories │ ├── InteropCustomAttributeFactory.cs # [Guid], [UnmanagedCallersOnly], [TypeMap], etc. │ ├── InteropMemberDefinitionFactory.cs # Properties, methods, lazy-init patterns +│ ├── InteropMethodDefinitionFactory.*.cs # Per-interface method body factories (20+ partials) +│ ├── InteropTypeDefinitionFactory.SzArrayMarshaller.cs # SZ array marshaller type emission +│ ├── InteropTypeDefinitionFactory.SzArrayElementMarshaller.cs # SZ array element marshaller emission +│ ├── InteropTypeDefinitionFactory.IEnumeratorElementMarshaller.cs # Collection element marshaller emission +│ ├── InteropTypeDefinitionFactory.IReadOnlyCollectionKeyValuePair2.cs # IReadOnlyCollection> emission │ ├── InteropUtf8NameFactory.cs # ABI-prefixed namespace/type name generation │ ├── WellKnownTypeDefinitionFactory.cs # IUnknownVftbl, IInspectableVftbl, DelegateVftbl -│ └── WellKnownMemberDefinitionFactory.cs # IID/Vtable property definitions +│ ├── WellKnownMemberDefinitionFactory.cs # IID/Vtable property definitions +│ └── ... (8 more) # Various well-known and utility factories ├── Fixups/ # Post-emit IL cleanup │ ├── InteropMethodFixup.cs # Abstract base with label redirect helpers │ ├── InteropMethodFixup.RemoveLeftoverNopAfterLeave.cs # ECMA-335 compliance @@ -389,13 +395,16 @@ For each discovered type, the generator emits some or all of these components: - **ComWrappersMarshallerAttribute** — For opaque object marshalling - **Proxy type** — `[WindowsRuntimeClassName]` + marshaller attribute for type map registration - **Type map attributes** — `[TypeMap<*TypeMapGroup>]` attributes for all three type map groups +- **Element marshaller type** (when applicable) — Implements a `IWindowsRuntime*ElementMarshaller` interface from the runtime, providing per-element conversion logic for collection `GetMany` CCW methods. Emitted by the `IEnumerator` builder and reused by `IList` and `IReadOnlyList` builders. Not needed for blittable, `object`, `string`, `Type`, or `Exception` element types (these use specialized direct paths). For additional details on generic interface code generation, see `references/marshalling-generic-interfaces.md`. **Per SZ array type (e.g., `string[]`):** -- **Array marshaller** — `ConvertToManaged`/`ConvertToUnmanaged`/`CopyToManaged`/`CopyToUnmanaged`/`Free` +- **Array element marshaller type** (when applicable) — Implements a `IWindowsRuntime*ArrayElementMarshaller` interface from the runtime, providing bidirectional per-element conversion. Emitted for non-trivial element types: `KeyValuePair`, `Nullable`, managed value types, unmanaged value types, and reference types. Not needed for blittable value types, `object`, `string`, `Type`, or `Exception` (these use specialized direct marshallers). +- **Array marshaller** — `ConvertToManaged`/`ConvertToUnmanaged`/`CopyToManaged`/`CopyToUnmanaged`/`Free`. Delegates to the runtime's generic array marshaller classes, passing the generated element marshaller type as a generic type argument when applicable. - **Array ComWrappers callback** — `IWindowsRuntimeArrayComWrappersCallback` implementation - **Array impl type** — `IReferenceArray` vtable with `get_Value` CCW method +- **Interface entries impl type** — CCW interface entries for the array type - **Proxy + marshaller attribute** — For opaque object marshalling For additional details on array code generation, see `references/marshalling-arrays.md`. @@ -433,7 +442,7 @@ Builders construct complete type definitions with IL method bodies. They use a * | `InteropTypeDefinitionBuilder.cs` | Core: IID, NativeObject, ComWrappersCallback | | `.Delegate.cs` | Generic delegate marshalling (vtable, native type, marshaller, impl, event source) | | `.EventSource.cs` | Event source types for EventHandler and collection changed events | -| `.IEnumerator1.cs` | `IEnumerator` methods impl, marshaller, CCW | +| `.IEnumerator1.cs` | `IEnumerator` methods impl, marshaller, CCW, element marshaller | | `.IEnumerable1.cs` | `IEnumerable` methods impl, marshaller, CCW | | `.IList1.cs` | `IList` full interface marshalling | | `.IReadOnlyList1.cs` | `IReadOnlyList` full interface marshalling | @@ -449,7 +458,7 @@ Builders construct complete type definitions with IL method bodies. They use a * | `.ICollectionKeyValuePair2.cs` | `ICollection>` marshalling | | `.IReadOnlyCollectionKeyValuePair2.cs` | `IReadOnlyCollection>` marshalling | | `.SzArray.cs` | `T[]` array marshallers (element-type-specific strategies) | -| `.UserDefinedType.cs` | CCW interface entries + vtable emission | +| `.UserDefinedType.cs` | CCW interface entries + vtable | **Other builders:** - **`WindowsRuntimeTypeHierarchyBuilder`** — Emits a frozen hash table (keys/values/buckets as RVA static data) for O(1) runtime class name → type lookup @@ -463,6 +472,12 @@ Factories create individual metadata elements (types, methods, properties, attri - **`InteropCustomAttributeFactory`** — Creates `CustomAttribute` instances: `[Guid]`, `[UnmanagedCallersOnly]`, `[DisableRuntimeMarshalling]`, `[TypeMap<*>]`, `[AttributeUsage]`, `[IgnoresAccessChecksTo]`, `[AssemblyMetadata]` - **`InteropMemberDefinitionFactory`** — Creates property/method definitions. Key pattern: `LazyVolatileReferenceDefaultConstructorReadOnlyProperty()` — a lazy-initialized static property using `Interlocked.CompareExchange` for thread-safe initialization +- **`InteropMethodDefinitionFactory`** — Per-interface method body factories (20+ partial files, e.g., `.IEnumerator1Impl.cs`, `.IList1Impl.cs`, `.IReadOnlyList1Impl.cs`). Generate IL method bodies for CCW and RCW methods. Key methods like `GetMany()` consume element marshaller types via `emitState.LookupTypeDefinition(elementType, "ElementMarshaller")`. +- **`InteropTypeDefinitionFactory`** — Per-category type definition factories (partial files): + - **`.SzArrayElementMarshaller.cs`** — Emits concrete element marshaller types for SZ arrays (one per element type category: `UnmanagedValueType`, `ManagedValueType`, `KeyValuePair`, `NullableValueType`, `ReferenceType`). Generated types implement `IWindowsRuntime*ArrayElementMarshaller` interfaces from the runtime. + - **`.IEnumeratorElementMarshaller.cs`** — Emits concrete element marshaller types for collection interfaces (same 5 categories). Generated types implement `IWindowsRuntime*ElementMarshaller` interfaces. Tracked in emit state for reuse by `IList` and `IReadOnlyList` method factories. + - **`.SzArrayMarshaller.cs`** — Emits array marshaller types that forward to runtime array marshaller classes, passing the element marshaller type as a generic argument. + - **`.IReadOnlyCollectionKeyValuePair2.cs`** — Emits `IReadOnlyCollection>` types. - **`WellKnownTypeDefinitionFactory`** — Creates fundamental vtable struct types (`IUnknownVftbl`, `IInspectableVftbl`, `DelegateVftbl`) with sequential layout and unmanaged function pointer fields - **`WellKnownMemberDefinitionFactory`** — Creates IID properties (backed by RVA static data containing GUID bytes) and Vtable properties - **`InteropUtf8NameFactory`** — Generates ABI-prefixed type/namespace names in `Utf8String` format diff --git a/.github/skills/interop-generator/references/marshalling-arrays.md b/.github/skills/interop-generator/references/marshalling-arrays.md index dde7305af1..4c2bab60bb 100644 --- a/.github/skills/interop-generator/references/marshalling-arrays.md +++ b/.github/skills/interop-generator/references/marshalling-arrays.md @@ -164,3 +164,102 @@ public static class Array; ``` This code allows fully supporting arrays of any types, in an efficient manner, and in a trimmable way. The design of the marshaller types also allows marshalling stubs for both RCW methods and CCW methods to leverage things such as `stackalloc` and/or array pooling, where needed. For instance, passing a "FillArray" to a native method from an RCW marshalling stub doesn't need to actually allocate the marshalled array in native memory. Rather, it can do a "stackalloc or rent from pool" for the array of marshalled values, pass that to native, and then use `CopyToManaged` from that to copy the values back to the destination managed `Span`. And of course, blittable types don't even need this at all, and can directly pass pinned buffers to native methods 🚀 + +## Element marshallers + +For non-trivial element types, array marshalling uses an **element marshaller pattern** — a separate generated type that handles bidirectional conversion of individual elements. The runtime's generic array marshaller classes accept the element marshaller as a `TElementMarshaller` generic type parameter, enabling the conversion logic to be resolved statically (via static abstract interface dispatch) with zero virtual overhead. + +### Why element marshallers exist + +Array marshalling involves iterating over elements and converting each one between its managed and ABI representation. For simple cases (blittable value types, strings, `object`, `Type`, `Exception`), dedicated array marshaller classes in the runtime handle this directly. But for types that require type-specific conversion logic — reference types, `KeyValuePair`, `Nullable`, managed value types, unmanaged value types — the runtime can't know the marshalling logic at compile time. Element marshallers bridge this gap: + +- The **runtime** defines the array marshalling algorithm (allocate, iterate, convert, free) in generic classes +- The **interop generator** provides the per-type conversion strategy by emitting concrete element marshaller types +- The element marshaller type is passed as a generic type parameter, enabling **zero-cost abstraction** — the JIT/AOT compiler can inline the element conversion calls + +### Runtime interfaces (in `WinRT.Runtime.dll`) + +Five element marshaller interfaces are defined in `WindowsRuntime.InteropServices.Marshalling` (under `InteropServices/Marshalling/SzArrays/`): + +| Interface | Element type | Members | +|-----------|-------------|---------| +| `IWindowsRuntimeReferenceTypeArrayElementMarshaller` | Reference types | `ConvertToUnmanaged(T?)`, `ConvertToManaged(void*)` | +| `IWindowsRuntimeManagedValueTypeArrayElementMarshaller` | Managed value types (ABI needs cleanup) | `ConvertToUnmanaged(T)`, `ConvertToManaged(TAbi)`, `Dispose(TAbi)` | +| `IWindowsRuntimeUnmanagedValueTypeArrayElementMarshaller` | Unmanaged value types | `ConvertToUnmanaged(T)`, `ConvertToManaged(TAbi)` | +| `IWindowsRuntimeKeyValuePairTypeArrayElementMarshaller` | `KeyValuePair` | `ConvertToUnmanaged(KeyValuePair)`, `ConvertToManaged(void*)` | +| `IWindowsRuntimeNullableTypeArrayElementMarshaller` | `Nullable` | `ConvertToUnmanaged(T?)`, `ConvertToManaged(void*)` | + +All are `static abstract` interfaces (using the C# static abstract member pattern), `[Obsolete]`, and `[EditorBrowsable(Never)]` — they are implementation details consumed only by generated code. + +### Runtime array marshaller classes (in `WinRT.Runtime.dll`) + +Each element marshaller interface has a corresponding generic array marshaller class that takes `TElementMarshaller` as a generic type parameter on its methods: + +| Array marshaller class | Element marshaller constraint | Methods | +|----------------------|------------------------------|---------| +| `WindowsRuntimeReferenceTypeArrayMarshaller` | `IWindowsRuntimeReferenceTypeArrayElementMarshaller` | `ConvertToUnmanaged`, `ConvertToManaged`, `CopyToUnmanaged`, `CopyToManaged` | +| `WindowsRuntimeManagedValueTypeArrayMarshaller` | `IWindowsRuntimeManagedValueTypeArrayElementMarshaller` | Same + `Dispose`, `Free` | +| `WindowsRuntimeUnmanagedValueTypeArrayMarshaller` | `IWindowsRuntimeUnmanagedValueTypeArrayElementMarshaller` | `ConvertToUnmanaged`, `ConvertToManaged`, `CopyToUnmanaged`, `CopyToManaged` | +| `WindowsRuntimeKeyValuePairTypeArrayMarshaller` | `IWindowsRuntimeKeyValuePairTypeArrayElementMarshaller` | Same 4 methods | +| `WindowsRuntimeNullableTypeArrayMarshaller` | `IWindowsRuntimeNullableTypeArrayElementMarshaller` | Same 4 methods | + +The remaining array marshaller classes do **not** use element marshallers because they handle element conversion directly: + +| Array marshaller class | Element type | Why no element marshaller | +|----------------------|-------------|--------------------------| +| `WindowsRuntimeBlittableValueTypeArrayMarshaller` | Blittable value types | Memory layout is identical; uses bulk copy | +| `HStringArrayMarshaller` | `string` | Specialized HSTRING fast path | +| `WindowsRuntimeObjectArrayMarshaller` | `object` | Uses `WindowsRuntimeObjectMarshaller` directly | +| `ExceptionArrayMarshaller` | `Exception` | Blittable ABI representation | +| `TypeArrayMarshaller` | `Type` | Specialized marshalling | + +### Generated element marshaller types (in `WinRT.Interop.dll`) + +The interop generator emits one concrete element marshaller type per element type that needs one. Generation is handled by `InteropTypeDefinitionFactory.SzArrayElementMarshaller`, with separate factory methods per category: `ReferenceType()`, `ManagedValueType()`, `UnmanagedValueType()`, `KeyValuePair()`, `NullableValueType()`. + +**Type shape:** +- **Name**: `ElementMarshaller` (e.g., `ElementMarshaller`) +- **Kind**: `sealed struct` (value type) for value-type elements, `abstract class` for reference-type elements +- **Interface**: Implements the matching `IWindowsRuntime*ArrayElementMarshaller` interface +- **Members**: `ConvertToUnmanaged(...)`, `ConvertToManaged(...)`, and optionally `Dispose(...)` — all emitted as stub methods with `nop` markers, rewritten during the two-pass IL emit phase + +**Selection logic** (in `InteropTypeDefinitionBuilder.SzArray.Marshaller()`): + +| Element type category | Element marshaller factory | Array marshaller factory | +|----------------------|---------------------------|------------------------| +| Blittable value type | *(none)* | `SzArrayMarshaller.BlittableValueType()` | +| `KeyValuePair` | `SzArrayElementMarshaller.KeyValuePair()` | `SzArrayMarshaller.KeyValuePair()` | +| `Nullable` | `SzArrayElementMarshaller.NullableValueType()` | `SzArrayMarshaller.NullableValueType()` | +| Managed value type | `SzArrayElementMarshaller.ManagedValueType()` | `SzArrayMarshaller.ManagedValueType()` | +| Other value type | `SzArrayElementMarshaller.UnmanagedValueType()` | `SzArrayMarshaller.UnmanagedValueType()` | +| `object` | *(none)* | `SzArrayMarshaller.Object()` | +| `string` | *(none)* | `SzArrayMarshaller.String()` | +| `System.Type` | *(none)* | `SzArrayMarshaller.Type()` | +| `System.Exception` | *(none)* | `SzArrayMarshaller.Exception()` | +| Other reference type | `SzArrayElementMarshaller.ReferenceType()` | `SzArrayMarshaller.ReferenceType()` | + +When an element marshaller is generated, the array marshaller factory receives it and passes the element marshaller type as a generic argument when calling the runtime array marshaller methods. For example, a generated `<<#Windows>JsonObject>ArrayMarshaller.ConvertToManaged(...)` would call `WindowsRuntimeReferenceTypeArrayMarshaller.ConvertToManaged<<<#Windows>JsonObject>ArrayElementMarshaller>(size, value)`. + +### Example: generated reference-type array element marshaller + +Array element marshallers are only generated for Windows Runtime types (i.e., types projected from `.winmd` metadata), because only those types support `IReferenceArray` boxing. User-defined managed types like `MyApp.MyViewModel` can get CCW marshalling code if they implement Windows Runtime interfaces, but they would not get array element marshallers. + +For a Windows Runtime type like `Windows.Data.Json.JsonObject`, the interop generator emits (in namespace `ABI.Windows.Data.Json`): + +```csharp +public abstract class <<#Windows>JsonObject>ArrayElementMarshaller + : IWindowsRuntimeReferenceTypeArrayElementMarshaller +{ + public static WindowsRuntimeObjectReferenceValue ConvertToUnmanaged(JsonObject? value) + { + return ABI.Windows.Data.Json.JsonObjectMarshaller.ConvertToUnmanaged(value); + } + + public static JsonObject? ConvertToManaged(void* value) + { + return ABI.Windows.Data.Json.JsonObjectMarshaller.ConvertToManaged(value); + } +} +``` + +The name follows the standard mangling scheme: the SZ array type `JsonObject[]` produces `<<#Windows>JsonObject>Array` (see `references/name-mangling-scheme.md`), and the `ElementMarshaller` suffix is appended. The element marshaller methods forward to the type's existing marshaller from the generated projection assembly. The method bodies are emitted as stub `nop` markers during pass 1 and filled in with the appropriate marshalling IL during pass 2 (see the two-pass IL emit section in the main skill document). diff --git a/.github/skills/interop-generator/references/marshalling-generic-interfaces.md b/.github/skills/interop-generator/references/marshalling-generic-interfaces.md index 0f3c6b8ac2..3797ea4868 100644 --- a/.github/skills/interop-generator/references/marshalling-generic-interfaces.md +++ b/.github/skills/interop-generator/references/marshalling-generic-interfaces.md @@ -272,6 +272,46 @@ The `GetInstance` method efficiently retrieves or creates adapter instances usin For the `GetMany` method and other type-specific operations, extension methods on `IEnumeratorAdapter` are provided in the runtime, with specialized overloads for different element type categories (blittable value types, managed value types, unmanaged value types, key-value pairs, reference types, strings, objects, etc.). +### Collection element marshallers + +For `GetMany` CCW methods, the runtime's collection adapter extension types (`IEnumeratorAdapterExtensions`, `IListAdapterExtensions`, `IReadOnlyListAdapterExtensions`) accept a `TElementMarshaller` generic type parameter to perform per-element managed → ABI conversion. This follows the same strategy pattern used for SZ array element marshallers (see `references/marshalling-arrays.md`), but with a key difference: **collection element marshallers are one-way** (managed → ABI only), whereas array element marshallers are bidirectional. + +**Runtime interfaces** (in `WinRT.Runtime.dll`, under `InteropServices/Marshalling/Collections/`): + +| Interface | Element type | Members | +|-----------|-------------|---------| +| `IWindowsRuntimeReferenceTypeElementMarshaller` | Reference types | `ConvertToUnmanaged(T?)` | +| `IWindowsRuntimeManagedValueTypeElementMarshaller` | Managed value types | `ConvertToUnmanaged(T)`, `Dispose(TAbi)` | +| `IWindowsRuntimeUnmanagedValueTypeElementMarshaller` | Unmanaged value types | `ConvertToUnmanaged(T)` | +| `IWindowsRuntimeKeyValuePairTypeElementMarshaller` | `KeyValuePair` | `ConvertToUnmanaged(KeyValuePair)` | +| `IWindowsRuntimeNullableTypeElementMarshaller` | `Nullable` | `ConvertToUnmanaged(T?)` | + +These are `static abstract` interfaces, `[Obsolete]`, and `[EditorBrowsable(Never)]` — implementation details consumed only by generated code. + +**Runtime consumer example** — a `GetMany` extension method on `IEnumeratorAdapterExtensions`: + +```csharp +public unsafe uint GetMany(uint itemsSize, void** items) + where TElementMarshaller : IWindowsRuntimeReferenceTypeElementMarshaller; +``` + +Similar overloads exist constrained to each of the other element marshaller interfaces. `IListAdapterExtensions` and `IReadOnlyListAdapterExtensions` also provide matching `GetMany` methods. + +**Generated element marshaller types** (in `WinRT.Interop.dll`): + +The interop generator emits concrete element marshaller types via `InteropTypeDefinitionFactory.IEnumeratorElementMarshaller`, with the same 5 factory methods as for SZ arrays: `ReferenceType()`, `ManagedValueType()`, `UnmanagedValueType()`, `KeyValuePair()`, `NullableValueType()`. The generated types: + +- Implement the matching `IWindowsRuntime*ElementMarshaller` interface +- Are emitted as `sealed struct` (value types) or `abstract class` (reference types), same as for array element marshallers +- Contain a `ConvertToUnmanaged(...)` stub method (rewritten during pass 2) +- Are named `ElementMarshaller` +- Are **emitted by the `IEnumerator` builder** and tracked in emit state via `emitState.TrackTypeDefinition(elementMarshallerType, elementType, "ElementMarshaller")` +- Are **reused by the `IList` and `IReadOnlyList` method factories** via `emitState.LookupTypeDefinition(elementType, "ElementMarshaller")` — they are not re-emitted + +The same element type categories that skip element marshallers for arrays also skip them for collections: blittable value types, `object`, `string`, `Type`, and `Exception` use specialized direct paths. + +**Why one-way?** Collection `GetMany` methods copy items from a managed collection *out* to a native buffer. The reverse direction (native → managed) is handled by the RCW `Methods` types (e.g., `IIteratorMethodsImpl.Current`), which use the full two-pass rewrite pipeline directly — they don't go through element marshallers. + We can now move on to the actual CCW implementation. As with all interfaces, we'll need a vtable: ```csharp diff --git a/.github/skills/update-interop-generator-instructions/SKILL.md b/.github/skills/update-interop-generator-instructions/SKILL.md index c4dc085ee2..9ad779d4ed 100644 --- a/.github/skills/update-interop-generator-instructions/SKILL.md +++ b/.github/skills/update-interop-generator-instructions/SKILL.md @@ -71,7 +71,13 @@ Launch an explore agent to verify: - **Define*Types() method list** matches the actual methods in `InteropGenerator.Emit.cs` - **Builder partial files** in `Builders/` are complete (check for added or removed partials) - **What gets generated** per type category is still accurate -- **Factory classes** in `Factories/` are current +- **Factory classes** in `Factories/` are current, including: + - `InteropTypeDefinitionFactory` partials (SzArrayMarshaller, SzArrayElementMarshaller, IEnumeratorElementMarshaller, IReadOnlyCollectionKeyValuePair2) + - `InteropMethodDefinitionFactory` partials (per-interface method body factories) +- **Element marshaller generation** is accurately described: + - SZ array element marshallers: which element categories get them, factory methods, runtime interfaces + - Collection element marshallers: emission by IEnumerator1 builder, reuse by IList1/IReadOnlyList1, runtime interfaces + - Selection logic table (which element types use direct paths vs element marshallers) - **Dynamic custom-mapped types** are current (check builder partial files) ### Step 8: verify two-pass IL and fixups @@ -143,7 +149,10 @@ If the changes to the interop generator are significant enough to affect the des - `references/marshalling-arrays.md` — Array marshalling design - `references/name-mangling-scheme.md` — Name mangling scheme for generated interop types -These docs describe the *design* of the generated code patterns. If the actual generated code has diverged from what these docs describe (e.g., new types generated, changed API patterns, renamed infrastructure types), update the docs to match. +These docs describe the *design* of the generated code patterns. If the actual generated code has diverged from what these docs describe (e.g., new types generated, changed API patterns, renamed infrastructure types), update the docs to match. In particular: + +- `marshalling-arrays.md` includes the element marshaller infrastructure (runtime interfaces, runtime array marshaller classes, generated element marshaller types, selection logic table) +- `marshalling-generic-interfaces.md` includes the collection element marshaller infrastructure (runtime interfaces, GetMany adapter extension methods, generated element marshaller types, emission/reuse pattern) ### Step 14: update this skill if needed