Skip to content

superkabuki/threefive_is_scte35

BranchesTags

Repository files navigation

Try threefive.js

[ threefive ]

https://github.com/superkabuki/threefive

threefive is the industry leading SCTE-35 tool.

Need to inject SCTE-35 into HLS? X9k3.

  • Decodes SCTE-35 from MPEGTS✔ Base64✔ Bytes✔ DASH✔ Hex ✔ HLS✔ Integers✔ JSON✔ XML✔ XML+Binary✔

  • Encodes SCTE-35 to MPEGTS✔ Base64✔ Bytes✔ Hex✔ Integers✔ JSON✔ XML✔ XML+Binary✔

Make threefive run twice as fast with python3.11 and python3.14

  • Side Note: threefive is fast enough to parse 13k video as is now, this is just me trying to see how fast I can make threefive run on python3.
  • This parses SCTE-35 from an MPEGTS stream.
  • This is the super funk version
    • python3.11 runs almost twice as fast
    • python3.14 runs almost three times faster
  • Test it
    • time python3 mps.py video.ts
    • compare with time threefive video.ts
  • Expect a multiprocessing option to be included in the next threefive release.
  • mps.py
"""
mps.py -  testing multiprocess performance of threefive on python 3.11 and 3.14
"""

import multiprocessing
import sys
from functools import partial
from threefive import Stream, reader

PKTSIZE = 188
CHUNKSIZE = PKTSIZE * 7700
POOLSIZE = 6

multiprocessing.set_start_method = "spawn"


class MPStream:
    def __init__(self, filepath):
        self.filepath = filepath
        self.pids = None
        self.maps = None
        self.init_pool()

    def init_pool(self):
        """
        init_pool discovers the stucture of an
        MPEGTS stream to prime
        Stream instances in the pool
        """
        stp = Stream(self.filepath)
        stp.show()
        stp.maps.prgm_pts = {}
        stp.maps.partial = {}
        self.pids = stp.pids
        self.maps = stp.maps

    def chunk_parser(self, chunk):
        """
        chunk_parser parse a chunk
        """
        st = Stream(None)
        st.pids = self.pids
        st.maps = self.maps
        return [cue for cue in [st._parse(pkt) for pkt in st.packetize(chunk)] if cue]

    def chunker(self):
        """
        chunker video chunk generator
        """
        with reader(self.filepath) as r:
            for chunk in iter(partial(r.read, CHUNKSIZE), b""):
                yield chunk

    def run(self):
        """
        run create pool and parse mpegts stream
        """
        with multiprocessing.Pool(
            POOLSIZE,
        ) as pool:
            results = pool.imap(self.chunk_parser, self.chunker(), chunksize=10)
            _ = [cue.show() for cues in results for cue in cues]


if __name__ == "__main__":
    mps = MPStream(sys.argv[1])
    mps.run()

Tip of the week.

Q. How do I get a list of all of the SCTE-35 cues in a stream?

A. Call Stream.decode_next(), it's a python generator.

from threefive import Stream

strm=Stream('https://example.com/video.ts')
list_o_cues=[]
for cue in strm.decode_next():    
	list_o_cues.append(cue)       # these will be threefive.Cue instances

# then you can do stuff like

_=[cue.show() for cue in list_o_cues]

for cue in list_o_cues:
	print(cue.xml())

[ News ]

[ Latest version is v3.0.83 ]

  • threefive cyclomatic complexity score is 1.9337094499294782 ( that's better than the Python standard library) .
  • threefive now has NO External Dependencies
    • SRT and AES support is now optional
  • threefive is fully python v3.14 compliant
  • No more setup tools! threefive now uses a toml file and a Makefile to generate packages,
    • I'm just trying to fit in with the cool python kids.
  • New the threefive cli tool has spun off several new cli tools. I had to split the cli up, the help was just way too long.
    • In addition to the threefive cli you also get:
      • scte35bump adjust scte-35 pts in mpegts streams
      • scte35fix change bin data streams back to scte-35
      • scte35hls parse scte-35 from hls tags and segments
      • scte35inject inject scte-35 packets into mpegts streams
      • gums (the Grande Unified unicast and Multicast Server)

[ Examples ]

[ Documentation ]

[Install]

  • python3 via pip
python3 -mpip install threefive
  • pypy3 via pip
pypy3 -mpip install threefive
  • To add SRT support
python3 -m pip install srtfu
  • To add Automatic AES decryption
python3 -mpip install pyaes
  • From the git repo
git clone https://github.com/superkabuki/scte35.git
cd threefive
make install
  • I've jazzed up the makefile to make it easier to install for different python versions and pypy3
git clone https://github.com/superkabuki/scte35.git
cd threefive

make install py3=pypy3

# OR

make install py3=python3.14

# works for any python in your path or use a full path if needed.

[Quick Start]

[CLI]

  • The default action is to read a SCTE-35 input and write a SCTE-35 output.

[ Parse SCTE-35 from MPEGTS ]

  • SCTE-35 Input: MPEGTS
  • Protocols: pipes, files, stdin, http(s), multicast,SRT and UDP.
  • SCTE-35 Output: JSON (default) base64, bytes, hex, int, xml, and xmlbin.
SCTE-35 Input Protocol SCTE-35 Output Command
MPEGTS file JSON threefive video.ts
. https base64 threefive https://example.com/video.ts base64
. multicast bytes threefive udp://@235.3.5:3535 bytes
. SRT hex threefive srt://1.2.3.4:4201 hex
. UDP int threefive udp://10.10.10.10:1011 int
. Pipe xml cat video.ts | threefive xml
. stdin xml+bin threefive xmlbin < video.ts

[ Parse SCTE-35 Cues ]

  • The default output is JSON
  • SCTE-35 Inputs: base64, hex, int, JSON,int,xml,and xmlbin.
  • SCTE-35 Outputs: base64, bytes, hex, int,JSON, xml, and xmlbin.
  • Any Input can be used with Any Output

Here are several examples.

SCTE-35 Input SCTE-35 Output Command
base64 JSON threefive '/DAWAAAAAAAAAP/wBQb+AKmKxwAACzuu2Q=='
. bytes threefive '/DAWAAAAAAAAAP/wBQb+AKmKxwAACzuu2Q==' bytes
. hex threefive '/DAWAAAAAAAAAP/wBQb+AKmKxwAACzuu2Q==' hex
. xml threefive '/DAWAAAAAAAAAP/wBQb+AKmKxwAACzuu2Q==' xml
hex JSON threefive 0xfc301600000000000000fff00506fe00a98ac700000b3baed9
. base64 threefive 0xfc301600000000000000fff00506fe00a98ac700000b3baed9 base64
. int threefive 0xfc301600000000000000fff00506fe00a98ac700000b3baed9 int
. xmlbin threefive 0xfc301600000000000000fff00506fe00a98ac700000b3baed9 xmlbin
int JSON threefive 1583008701074197245727019716796221242036302348025116111908569
. hex threefive 1583008701074197245727019716796221242036302348025116111908569 hex
. xml threefive 1583008701074197245727019716796221242036302348025116111908569 xml
JSON base64 threefive < json.json base64
. bytes threefive < json.json bytes
. xml threefive < json.json xml
xml JSON threefive < xml.xml
xmlbin int threefive < xmlbin.xml int

[Additional functionality]

  • threefive has several additional features, mostly related to MPEGTS streams.
  • threefive has built in help, just type threefive help
  • This table shows how to use them.
Description How To Use
Inject SCTE35 packets threefive inject -i in.video -s sidecar.txt -o out.ts
Show raw SCTE35 packets threefive packets udp://@235.35.3.5:3535
Copy MPEGTS stream to stdout at realtime speed threefive rt input.ts
Create SCTE35 sidecar file threefive sidecar video.ts
Show streams in mpegts stream threefive show https://example.com/video.ts
Show iframes in mpegts stream threefive iframes srt://10.10.1.3:9000
Show PTS values from mpegts stream threefive pts udp://192.168.1.10:9000
Proxy the mpegts stream to stdout threefive proxy https://wexample.com/video.ts

Other tools

threefive also comes with:

scte35bump

  • bump adjusts SCTE-35 PTS in an MPEGTS stream
$ scte35bump -h
usage: scte35bump [-h] [-i INFILE] [-o OUTFILE] [-s SECS]

options:
  -h, --help            show this help message and exit
  -i INFILE, --infile INFILE
                        Input source, stdin, file, http(s), udp, or multicast
                        mpegts [default: sys.stdin.buffer]
  -o OUTFILE, --outfile OUTFILE
                        Output file [default: sys.stdout.buffer]
  -s SECS, --secs SECS  Adjustment to apply to SCTE-35 Cues. [default: 0.0]

scte35bump is part of threefive.

gums

  • the Grande Udp Multicast Server
$ gums -h
usage: gums [-h] [-i INPUT] [-a ADDR] [-b BIND_ADDR] [-t TTL]

options:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        like "/home/a/vid.ts" or "https://futzu.com/xaa.ts"
                        [default: sys.stdin.buffer]
  -a ADDR, --addr ADDR  Destination IP:Port [default: 235.35.3.5:3535]
  -b BIND_ADDR, --bind_addr BIND_ADDR
                        Local IP to bind [default: 0.0.0.0]
  -t TTL, --ttl TTL     Multicast TTL (1 - 255) [default: 32]

gums is part of threefive.

scte35hls

  • parse HLS for SCTE-35. Supports all HLS SCTE-35 tags.
$ scte35hls -h

[ threefive hls ]

[ Help ]

    To display this help:
	scte35hls help

[ Input ]
    threefive hls takes an m3u8 URI as input.
    M3U8 formats supported:
        * master  ( When a master.m3u8 used,
                   threefive hls parses the first rendition it finds )
        * rendition
    Segment types supported:
    * AAC
    * AC3
    * MPEGTS
    *codecs:
        * video
            * mpeg2, h.264, h.265
        * audio
            * mpeg2, aac, ac3, mp3

scte35fix

  • when ffmpeg changes a SCTE-35 stream to bin data stream, scte35fix changes it back.
$ scte35fix -h

  scte35fix checks MPEGTS for SCTE-35 Streams
  that have been change to bin data (type 0x06)
  and changes them back to SCTE-35 (type 0x86) streams.
  Output files are created in the current directory
  and prefixed with 'sixfix-'.
  Only bin data streams containing SCTE-35 will be converted.
  Multiple files can be specified on the command line.
  Wild cards work too.

  Example Usage:
        scte35fix video.ts
        scte35fix video1.ts video2.ts
        scte35fix video*.ts
        scte35fix https://example.com/video.ts
        scte35fix srt://10.10.10.13:4201
scte35fix is part of threefive.

[XML]

  • XML New! updated 05/01/2025

[HLS]

[SCTE-35 As a Service]

  • Decode SCTE-35 without installing anything. If you can make an https request, you can use Sassy to decode SCTE-35. .

[Classes]

  • The python built in help is always the most up to date docs for the library.
a@fu:~/build7/threefive$ pypy3

>>>> from threefive import Stream
>>>> help(Stream)

[threefive now supports SRT]

  • ( You have to unmute the audio )
srt.mp4

[more]

Python3 vs Pypy3 running threefive

  • ( You have to unmute the audio )
pvp.mp4

Using the library

  • Let me show you how easy threefive is to use.

  • reading SCTE-35 xml from a file

a@fu:~/threefive$ pypy3
Python 3.9.16 (7.3.11+dfsg-2+deb12u3, Dec 30 2024, 22:36:23)
[PyPy 7.3.11 with GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> from threefive import reader
>>>> from threefive import Cue
>>>> data =reader('/home/a/xml.xml').read()
  • load it into a threefive.Cue instance
>>>> cue = Cue(data)
  • Show the data as JSON
>>>> cue.show()
{
    "info_section": {
        "table_id": "0xfc",
        "section_syntax_indicator": false,
        "private": false,
        "sap_type": "0x03",
        "sap_details": "No Sap Type",
        "section_length": 92,
        "protocol_version": 0,
        "encrypted_packet": false,
        "encryption_algorithm": 0,
        "pts_adjustment": 0.0,
        "cw_index": "0x00",
        "tier": "0x0fff",
        "splice_command_length": 15,
        "splice_command_type": 5,
        "descriptor_loop_length": 60,
        "crc": "0x7632935"
    },
    "command": {
        "command_length": 15,
        "command_type": 5,
        "name": "Splice Insert",
        "break_auto_return": false,
        "break_duration": 180.0,
        "splice_event_id": 1073743095,
        "splice_event_cancel_indicator": false,
        "out_of_network_indicator": true,
        "program_splice_flag": false,
        "duration_flag": true,
        "splice_immediate_flag": false,
        "event_id_compliance_flag": true,
        "unique_program_id": 1,
        "avail_num": 12,
        "avails_expected": 5
    },
    "descriptors": [
        {
            "tag": 0,
            "identifier": "CUEI",
            "name": "Avail Descriptor",
            "provider_avail_id": 12,
            "descriptor_length": 8
        },
        {
            "tag": 0,
            "identifier": "CUEI",
            "name": "Avail Descriptor",
            "provider_avail_id": 13,
            "descriptor_length": 8
        },
      

    ]
}
  • convert the data back to xml
>>>> print(cue.xml())
<scte35:SpliceInfoSection xmlns:scte35="https://scte.org/schemas/35"  ptsAdjustment="0" protocolVersion="0" sapType="3" tier="4095">
   <scte35:SpliceInsert spliceEventId="1073743095" spliceEventCancelIndicator="false" spliceImmediateFlag="false" eventIdComplianceFlag="true" availNum="12" availsExpected="5" outOfNetworkIndicator="true" uniqueProgramId="1">
      <scte35:BreakDuration autoReturn="false" duration="16200000"/>
   </scte35:SpliceInsert>
   <scte35:AvailDescriptor providerAvailId="12"/>
   <scte35:AvailDescriptor providerAvailId="13"/>
   <scte35:AvailDescriptor providerAvailId="14"/>
   <scte35:AvailDescriptor providerAvailId="15"/>
   <scte35:AvailDescriptor providerAvailId="16"/>
   <scte35:AvailDescriptor providerAvailId="17"/>
</scte35:SpliceInfoSection>
  • convert to xml+binary
>>>> print(cue.xmlbin())
<scte35:Signal xmlns:scte35="https://scte.org/schemas/35">
    <scte35:Binary>/DBcAAAAAAAAAP/wDwVAAAT3f69+APcxQAABDAUAPAAIQ1VFSQAAAAwACENVRUkAAAANAAhDVUVJAAAADgAIQ1VFSQAAAA8ACENVRUkAAAAQAAhDVUVJAAAAEQdjKTU=</scte35:Binary>
</scte35:Signal>
  • convert to base64
>>>> print(cue.base64())
/DBcAAAAAAAAAP/wDwVAAAT3f69+APcxQAABDAUAPAAIQ1VFSQAAAAwACENVRUkAAAANAAhDVUVJAAAADgAIQ1VFSQAAAA8ACENVRUkAAAAQAAhDVUVJAAAAEQdjKTU=
  • convert to hex
>>>> print(cue.hex())
0xfc305c00000000000000fff00f05400004f77faf7e00f7314000010c05003c0008435545490000000c0008435545490000000d0008435545490000000e0008435545490000000f000843554549000000100008435545490000001107632935
  • show just the splice command
>>>> cue.command.show()
{
    "command_length": 15,
    "command_type": 5,
    "name": "Splice Insert",
    "break_auto_return": false,
    "break_duration": 180.0,
    "splice_event_id": 1073743095,
    "splice_event_cancel_indicator": false,
    "out_of_network_indicator": true,
    "program_splice_flag": false,
    "duration_flag": true,
    "splice_immediate_flag": false,
    "event_id_compliance_flag": true,
    "unique_program_id": 1,
    "avail_num": 12,
    "avails_expected": 5
}
  • edit the break duration
>>>> cue.command.break_duration=30
>>>> cue.command.show()
{
    "command_length": 15,
    "command_type": 5,
    "name": "Splice Insert",
    "break_auto_return": false,
    "break_duration": 30,
    "splice_event_id": 1073743095,
    "splice_event_cancel_indicator": false,
    "out_of_network_indicator": true,
    "program_splice_flag": false,
    "duration_flag": true,
    "splice_immediate_flag": false,
    "event_id_compliance_flag": true,
    "unique_program_id": 1,
    "avail_num": 12,
    "avails_expected": 5
}
  • re-encode to base64 with the new duration
>>>> cue.base64()
'/DBcAAAAAAAAAP/wDwVAAAT3f69+ACky4AABDAUAPAAIQ1VFSQAAAAwACENVRUkAAAANAAhDVUVJAAAADgAIQ1VFSQAAAA8ACENVRUkAAAAQAAhDVUVJAAAAEe1FB6g='
  • re-encode to xml with the new duration
>>>> print(cue.xml())
<scte35:SpliceInfoSection xmlns:scte35="https://scte.org/schemas/35"  ptsAdjustment="0" protocolVersion="0" sapType="3" tier="4095">
   <scte35:SpliceInsert spliceEventId="1073743095" spliceEventCancelIndicator="false" spliceImmediateFlag="false" eventIdComplianceFlag="true" availNum="12" availsExpected="5" outOfNetworkIndicator="true" uniqueProgramId="1">
      <scte35:BreakDuration autoReturn="false" duration="2700000"/>
   </scte35:SpliceInsert>
   <scte35:AvailDescriptor providerAvailId="12"/>
   <scte35:AvailDescriptor providerAvailId="13"/>
   <scte35:AvailDescriptor providerAvailId="14"/>
   <scte35:AvailDescriptor providerAvailId="15"/>
   <scte35:AvailDescriptor providerAvailId="16"/>
   <scte35:AvailDescriptor providerAvailId="17"/>
</scte35:SpliceInfoSection>
  • show just the descriptors
>>>> _ = [d.show() for d in cue.descriptors]
{
    "tag": 0,
    "identifier": "CUEI",
    "name": "Avail Descriptor",
    "provider_avail_id": 12,
    "descriptor_length": 8
}
{
    "tag": 0,
    "identifier": "CUEI",
    "name": "Avail Descriptor",
    "provider_avail_id": 13,
    "descriptor_length": 8
}
{
    "tag": 0,
    "identifier": "CUEI",
    "name": "Avail Descriptor",
    "provider_avail_id": 14,
    "descriptor_length": 8
}
{
    "tag": 0,
    "identifier": "CUEI",
    "name": "Avail Descriptor",
    "provider_avail_id": 15,
    "descriptor_length": 8
}
{
    "tag": 0,
    "identifier": "CUEI",
    "name": "Avail Descriptor",
    "provider_avail_id": 16,
    "descriptor_length": 8
}
{
    "tag": 0,
    "identifier": "CUEI",
    "name": "Avail Descriptor",
    "provider_avail_id": 17,
    "descriptor_length": 8
}
  • pop off the last descriptor and re-encode to xml
>>>> cue.descriptors.pop()
{'tag': 0, 'identifier': 'CUEI', 'name': 'Avail Descriptor', 'private_data': None, 'provider_avail_id': 17, 'descriptor_length': 8}
>>>> print(cue.xml())
<scte35:SpliceInfoSection xmlns:scte35="https://scte.org/schemas/35"  ptsAdjustment="0" protocolVersion="0" sapType="3" tier="4095">
   <scte35:SpliceInsert spliceEventId="1073743095" spliceEventCancelIndicator="false" spliceImmediateFlag="false" eventIdComplianceFlag="true" availNum="12" availsExpected="5" outOfNetworkIndicator="true" uniqueProgramId="1">
      <scte35:BreakDuration autoReturn="false" duration="2700000"/>
   </scte35:SpliceInsert>
   <scte35:AvailDescriptor providerAvailId="12"/>
   <scte35:AvailDescriptor providerAvailId="13"/>
   <scte35:AvailDescriptor providerAvailId="14"/>
   <scte35:AvailDescriptor providerAvailId="15"/>
   <scte35:AvailDescriptor providerAvailId="16"/>
</scte35:SpliceInfoSection>

[ The Cli tool ]

The cli tool installs automatically with pip or the Makefile.

Inputs

  • Most inputs are auto-detected.
  • stdin is auto selected and auto detected.
  • SCTE-35 data is printed to stderr
  • stdout is used when piping video
  • mpegts can be specified by file name or URI.
threefive udp://@235.2.5.35:3535
  • If a file comtains a SCTE-35 cue as a string( base64,hex,int,json,or xml+bin), redirect the file contents.
  threefive < json.json
  • quoted strings(( base64,hex,int,json or xml+bin), can be passed directly on the command line as well.
threefive '/DAWAAAAAAAAAP/wBQb+ztd7owAAdIbbmw=='
Input Type Cli Example
Base64 threefive '/DAsAAAAAyiYAP/wCgUAAAABf1+ZmQEBABECD0NVRUkAAAAAf4ABADUAAC2XQZU='
Hex threefive 0xfc301600000000000000fff00506fed605225b0000b0b65f3b
HLS threefive hls https://example.com/master.m3u8
JSON threefive < json.json
Xmlbin js threefive < xmlbin.xml

Streams

Protocol Cli Example
File threefive video.ts
Http(s) threefive https://example.com/video.ts
Stdin threefive < video.ts
UDP Multicast threefive udp://@235.35.3.5:9999
UDP Unicast threefive udp://10.0.0.7:5555

Outputs

  • output type is determined by the key words base64, bytes, hex, int, json, and xmlbin.
  • json is the default.
  • Any input can be returned as any output
    • examples Base64 to Hex etc...)
Output Type Cli Example
Base 64 threefive 0xfc301600000000000000fff00506fed605225b0000b0b65f3b base64
Bytes threefive 0xfc301600000000000000fff00506fed605225b0000b0b65f3b bytes
Hex threefive '/DAsAAAAAyiYAP/wCgUAAAABf1+ZmQEBABECD0NVRUkAAAAAf4ABADUAAC2XQZU=' hex
Integer threefive '/DAsAAAAAyiYAP/wCgUAAAABf1+ZmQEBABECD0NVRUkAAAAAf4ABADUAAC2XQZU=' int
JSON threefive 0xfc301600000000000000fff00506fed605225b0000b0b65f3b json
Xml+bin threefive 0xfc301600000000000000fff00506fed605225b0000b0b65f3b xmlbin

Iframes

  • Show iframes PTS in an MPEGTS video
threefive iframes https://example.com/video.ts

packets

  • Print raw SCTE-35 packets from multicast mpegts video
threefive packets udp://@235.35.3.5:3535

proxy

  • Parse a https stream and write raw video to stdout
threefive proxy video.ts

pts

  • Print PTS from mpegts video
threefive pts video.ts

sidecar

  • Parse a stream, write pts,write SCTE-35 Cues to sidecar.txt
threefive sidecar video.ts

show

  • Probe mpegts video ( kind of like ffprobe )
 threefive show video.ts

version

  • Show version
 threefive version

help

  • Help
 threefive help

iodisco.com/scte35

image

Install |SCTE-35 Cli | Cue Class | Stream Class | Online SCTE-35 Parser | SCTE-35 Examples | SCTE-35 XML and More XML | threefive runs Four Times Faster on pypy3

About

threefive, nobody does SCTE-35 better. Nobody.

iodisco.com/scte35

Topics

video python3 adtech mpegts video-streaming hls-live-streaming scte35 scte-35 threefive hls-scte35 adrianofdoom adbreak scte-214 scte-35-hls fu-based-technology udp4life splice-fu

Resources

Readme

License

GPL-2.0 license
Activity

Stars

19 stars

Watchers

1 watching

Forks

4 forks
Report repository

Releases

10 tags

Contributors

Languages