Skip to content

Preserve input time base in concatenate_video_files function#2160

Closed
irisTa56 wants to merge 2 commits intohuggingface:mainfrom
irisTa56:set-time-base-as-kwargs
Closed

Preserve input time base in concatenate_video_files function#2160
irisTa56 wants to merge 2 commits intohuggingface:mainfrom
irisTa56:set-time-base-as-kwargs

Conversation

@irisTa56
Copy link
Copy Markdown

@irisTa56 irisTa56 commented Oct 10, 2025

What this does

Fixes the second error of #2158.
Effectively reverts #2133.

We cannot set time_base to the created stream without PyAV-Org/PyAV#1934, so I updated the minimum version of PyAV.
I believe the updated requirement will not cause problems since requirements-*.txt already specifies av==15.0.0, but there might be use cases requiring PyAV<15 that I am not aware of.

How it was tested

With the following script (and some video file named test.mp4):

import shutil
from pathlib import Path

import av
from lerobot.datasets.video_utils import concatenate_video_files


def show_time_bases(path: Path):
    with av.open(str(path)) as container:
        stream = container.streams.video[0]
        print(f"{path}: time_base: {stream.time_base}")


test_data = Path("test.mp4")
assert test_data.exists()

# Videos will be concatenated into path0, step by step.
path0 = Path("path0.mp4")
path1 = Path("path1.mp4")
path2 = Path("path2.mp4")

for path in [path0, path1, path2]:
    path.unlink(missing_ok=True)
    shutil.copy(test_data, path)
    show_time_bases(path)

concatenate_video_files([path0, path1], path0)

print("After first concatenation:")
show_time_bases(path0)

concatenate_video_files([path0, path2], path0)

I got:

$ python test.py
path0.mp4: time_base: 1/10240
path1.mp4: time_base: 1/10240
path2.mp4: time_base: 1/10240
After first concatenation:
path0.mp4: time_base: 1/10240

I appreciate any advice about writing unit tests if needed.
I could not find existing tests for video_utils module.

How to checkout & try?

Same as the above.
Code at bf3c874 reproduces the problem as follows:

$ python test.py          
path0.mp4: time_base: 1/10240
path1.mp4: time_base: 1/10240
path2.mp4: time_base: 1/10240
After first concatenation:
path0.mp4: time_base: 1/90000
Application provided invalid, non monotonically increasing dts to muxer in stream 0: 513000 >= 59392
Traceback (most recent call last):
  File "/Users/takayuki/Documents/_repos/lerobot/_tmp/test.py", line 32, in <module>
    concatenate_video_files([path0, path2], path0)
  File "/Users/takayuki/Documents/_repos/lerobot/src/lerobot/datasets/video_utils.py", line 467, in concatenate_video_files
    output_container.mux(packet)
  File "av/container/output.py", line 375, in av.container.output.OutputContainer.mux
  File "av/container/output.py", line 399, in av.container.output.OutputContainer.mux_one
  File "av/container/core.pyx", line 303, in av.container.core.Container.err_check
  File "av/error.pyx", line 424, in av.error.err_check
av.error.ValueError: [Errno 22] Invalid argument: '/var/folders/12/z2mncvk52d75lblxw5wmpryw0000gn/T/tmpvto2jb97.mp4'; last error log: [mp4] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 513000 >= 59392

Copilot AI review requested due to automatic review settings October 10, 2025 05:54
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a video concatenation issue where the time base was not being preserved when concatenating video files, causing invalid DTS (decode timestamp) errors. The fix explicitly sets the time_base parameter when adding streams and updates the minimum PyAV version requirement to 15.0.0.

  • Preserves input time base in video concatenation to prevent timestamp conflicts
  • Updates minimum PyAV version requirement from 14.2.0 to 15.0.0 to support the time_base parameter
  • Adds detailed comments explaining the time base handling logic

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/lerobot/datasets/video_utils.py Added explicit time_base parameter and explanatory comments to preserve input stream time base during concatenation
pyproject.toml Updated minimum PyAV version from 14.2.0 to 15.0.0 to support the time_base parameter

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@irisTa56
Copy link
Copy Markdown
Author

Closing as this change has been included in #2100.

@irisTa56 irisTa56 closed this Oct 12, 2025
@irisTa56 irisTa56 deleted the set-time-base-as-kwargs branch October 20, 2025 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants