[Platform] Rework AssistantMessage

[valtzu]

Q	A
Bug fix?	no
New feature?	no
Docs?	todo
License	MIT

Rework AssistantMessage to better model the APIs and

1. Allow any platform-specific content that needs to be sent back to the server on next call
2. Retain original order of blocks/parts
3. Allow attaching metadata to blocks/parts instead of just the message.

• e.g. with Gemini, several parts in the assistant message can have thoughtSignatures

Looking for feedback before proceeding further as this change is quite enormous at this late stage

Before

new AssistantMessage(
  content: 'Hello',
  toolCalls: [
    new ToolCall('id1', 'function1', ['param1' => 'value1']),
    new ToolCall('id2', 'function2', ['param2' => 'value2']),
  ],
  thinkingContent: 'foo',
  thinkingSignature: 'bar',
),

After

new AssistantMessage(
  new MultiPartResult(
    new ThinkingResult('foo', 'bar'),
    new TextResult('Hello'),
    new ToolCallResult([new ToolCall('id1', 'function1', ['param1' => 'value1'])]),
    new ToolCallResult([new ToolCall('id2', 'function2', ['param2' => 'value2'])]),
  ),
),

[chr-hertel]

Thanks @valtzu, I like that proposal - and I'd skip the Collection - is more consistent with UserMessage.

cc @OskarStark @fabpot WDYT?

[OskarStark]

Sounds like a good and flexible approach

[fabpot]

I like it a lot!

[OskarStark]

@valtzu can you finish this one?

[valtzu]

@OskarStark Sorry I was afk the whole Easter; I'll try

Thanks all for comments

[valtzu]

Ok I think I may have taken a too big bite here. I'm struggling quite much on how to glue results & contents.

It could be easier to start from the lower layer (in a separate PR) and add some MultiPartResult/CollectionResult first to tackle #1053 – and only then think of how to map that to AssistantMessage

[valtzu]

I'm struggling quite much on how to glue results & contents.

After #1900 this could be adjusted to use ResultInterface instead of ContentInterface which would solve the issue.

[valtzu]

After #1900 this could be adjusted to use ResultInterface instead of ContentInterface which would solve the issue.

I tried this (pushed here) – it mostly works, but one of the issues is that as result objects contain raw results (AsyncResponse) they cannot be serialized.

Still somehow using ContentInterface felt more fitting (though much bigger change and something I cannot finish myself)

[chr-hertel]

yeah, having second thoughts as well. i agree that ContentInterface would be cleaner and maybe try to keep Message classes and Result classes decoupled

new AssistantMessage(
    new Thinking('foo', 'bar'),
    new Text('Hello'),
    new ToolCall('id1', 'function1', ['param1' => 'value1']),
    new new ToolCall('id2', 'function2', ['param2' => 'value2']),
),

[xprojects-de]

i think that´s great and i like it!