Skip to content

Conversation

@shan-nvidia
Copy link
Contributor

@shan-nvidia shan-nvidia commented Jan 6, 2026

What does this PR do ?

Unroll biencoder training data with multiple positive documents into records with single positive documents.

Changelog

  • Add a script that takes training data where each question has multiple positive documents and creates a new dataset where each (question, pos_doc) pair becomes its own record.
  • The script is manually tested.
  • The script has doc string as the documentation.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?

If you haven't finished some of the above items you can still open "Draft" PR.

Additional Information

  • Related to # (issue)

@copy-pr-bot
Copy link

copy-pr-bot bot commented Jan 6, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@shan-nvidia shan-nvidia changed the title Add unroll pos docs script for biencoder training data feat: Add unroll pos docs script for biencoder training data Jan 6, 2026
Signed-off-by: Steve Han <sthan@nvidia.com>
@shan-nvidia shan-nvidia force-pushed the steve/unroll_pos_docs_script branch from e780c7c to 2af51f7 Compare January 7, 2026 13:59
@akoumpa
Copy link
Contributor

akoumpa commented Jan 8, 2026

/ok to test 2af51f7

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
@akoumpa
Copy link
Contributor

akoumpa commented Jan 8, 2026

/ok to test 054980a

@akoumpa akoumpa enabled auto-merge (squash) January 8, 2026 16:34
@akoumpa akoumpa merged commit 7711f7b into main Jan 8, 2026
52 checks passed
@akoumpa akoumpa deleted the steve/unroll_pos_docs_script branch January 8, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants