Simple lightweight mp4/fmp4/mov/m4v reader/writer. Supports H261/H262/H263/MPEG4/H264/H265/H266/VP9/AV1/AV2/ProRes/Motion JPEG for video and AAC/Opus/MP3 (and MP2)/FLAC/ALAC/AC-3/E-AC-3/AMR/PCM (and G.711) for audio. No platform dependencies, easily portable cross-platform. It was designed to be a stream-in and stream-out solution for recording streams from IP cameras into MP4 and fragmented MP4.

The list of all supported boxes, entries and descriptors is here.

Each box says whether it is where its definition places it: box.ValidatePlacement() checks its parent against the containers of its definition in src/SharpISOBMFF/Definitions - a parent box type, the top of the file, a parent class, and of an encrypted or restricted sample entry, the format its frma names. It is for information only: a box elsewhere is read and written all the same. A box whose definition gives no container is not reported.

ToString() of a box, a class, a descriptor or a whole Container describes its fields as they are now - read, or changed since - a line each, as FFmpeg's trace_headers traces a bitstream: the field's position in bits, its name, its bits and its value, the fields and boxes in a box indented a level deeper, and the fields of a loop with their indices. A run of bytes kept in the file (StreamMarker), such as the data of mdat, is shown as ....

            'stts' TimeToSampleBox
0             size                       00000000000000000000000000100000 = 32
32            type                       01110011011101000111010001110011 = 1937011827 ('stts')
64            version                                            00000000 = 0
72            flags                              000000000000000000000000 = 0
96            entry_count                00000000000000000000000000000001 = 1
128           sample_count[0]            00000000000000000000000000000011 = 3
160           sample_delta[0]            00000000000000000000010000000000 = 1024

To parse an existing mp4/mov/m4v file, first you have to get the stream:

using (Stream inputFileStream = new BufferedStream(new FileStream("frag_bunny.mp4", FileMode.Open, FileAccess.Read, FileShare.Read)))
{
    ...
}

Create a new Container and call Read to get the in-memory representation of all the boxes.

var mp4 = new Container();
mp4.Read(new IsoStream(inputFileStream));    ...

To process this in-memory representation and read the audio/video samples, create a new VideoReader:

VideoReader videoReader = new VideoReader();
videoReader.Parse(mp4);

Now it is possible to get all the tracks from the video:

IEnumerable<ITrack> tracks = videoReader.GetTracks();

To read the first track samples, call:

uint trackID = tracks.First().TrackID;
MediaSample sample = videoReader.ReadSample(trackID);

Where trackID is the ID of the track you want to read.

To write MP4 into a file, you first have to create the file:

using (Stream output = new BufferedStream(new FileStream("bunny_out.mp4", FileMode.Create, FileAccess.Write, FileShare.Read)))
{
    ...
}

Next, create the builder depending upon the output format. Currently, there are two builders - FragmentedMp4Builder and Mp4Builder.

IMp4Builder outputBuilder = new Mp4Builder(new SingleStreamOutput(output));

For fragmented MP4 output, use FragmentedMp4Builder with the fragment duration in miliseconds:

// use fragment duration 2 seconds
IMp4Builder outputBuilder = new FragmentedMp4Builder(new SingleStreamOutput(output), 2000);

To write another format of the MP4 family - .m4v, .m4a, .m4b, .3gp, .3g2 or .mov - set FileFormat before adding the tracks. The ftyp then says what the file is, and the builder refuses a track the format cannot hold, such as video in an .m4a or a second audio track in a 3GP file of the Basic profile:

outputBuilder.FileFormat = Mp4FileFormat.M4A;

Add the H264 video track to the builder instance:

var videoTrack = new H264Track();
outputBuilder.AddTrack(videoTrack);

Add the AAC audio track to the builder instance:

var audioTrack = new AACTrack(2, 44100, 16);
outputBuilder.AddTrack(audioTrack);

Pass the track samples to the builder as follows:

byte[] nalu = ...;
outputBuilder.ProcessTrackSample(videoTrack.TrackID, nalu);
...
byte[] aac = ...;
outputBuilder.ProcessTrackSample(audioTrack.TrackID, aac);

ProcessTrackSample takes H.264/H.265/H.266 NAL units without their start codes. To pass the Annex B byte stream as an encoder hands it out - one or more NAL units, each behind a start code - use ProcessAnnexBTrackSample, which strips the start codes:

byte[] annexB = ...; // 00 00 00 01 67 ... 00 00 00 01 68 ... 00 00 01 65 ...
outputBuilder.ProcessAnnexBTrackSample(videoTrack.TrackID, annexB);

The other audio tracks are made of their codec's configuration, as an encoder hands it out or a file has it, and take a frame a sample:

var flac = new FlacTrack(FlacStreamHeader.Parse(header));     // 'fLaC' and the metadata blocks, STREAMINFO first
var alac = new AlacTrack(alacSpecificConfig);                   // the 24 byte magic cookie
var mp3 = new Mp3Track(2, 44100);                              // MPEG-1 or MPEG-2 audio, by the rate
var ac3 = new AC3Track(AC3SyncFrame.Parse(frame));             // a sync frame's syncinfo and bsi
var eac3 = new EAC3Track(dataRate, substreams);                 // 'dec3''s data rate and independent substreams

An MP3 encoder hands out one frame or several at a time: Mp3Track.ParseFrames splits them, a sample each.

PCM is made of its format and written as QuickTime's lpcm, as Apple's devices write it. It takes a block of frames at a time and writes a sample of each, as QuickTime has it. Reading it, QuickTime's lpcm, sowt, twos, raw , in24, in32, fl32 and fl64 and ISO's ipcm and fpcm are read, and ReadSamples reads a run of frames at once rather than a read of each:

var pcm = new PcmTrack(48000, 2, 16);                           // rate, channels, bits; isFloat and isLittleEndian optional
outputBuilder.ProcessTrackSample(pcm.TrackID, block, block.Length / pcm.BytesPerFrame);
var run = reader.ReadSamples(trackID, 4800);                    // up to 4800 frames, their duration together

When done, call FinalizeMedia to create the video file:

outputBuilder.FinalizeMedia();

There is an IMp4Logger interface exposed on the Mp4Builder, FragmentedMp4Builder, VideoReader and ImageReader instances where you can supply your own logger implementation, or use an existing one:

mp4Builder.Logger = new ConsoleMp4Logger();

You can also enable/disable different trace levels like:

mp4Builder.Logger.IsWarnEnabled = false;

The published conformance suites for H.264, H.265, H.266 and AV1 can be downloaded for testing the parsers against:

.\DownloadConformance.ps1                            # the core sets for all four codecs, about 4.8 GB
.\DownloadConformance.ps1 -Codec H265, H266          # only some codecs
.\DownloadConformance.ps1 -IncludeSvc -IncludeArgon  # also H.264 SVC (12.9 GB) and AV1 Argon Streams (7 GB)

They go into the conformance folder, which git ignores. Running the script again skips what is already there. The reference decoder's output - decoded pictures, traces and logs, which make the H.264 suites several times their download size - is left out unless -KeepReference is given. It works in Windows PowerShell 5.1 and PowerShell 7.

Huge inspiration for this project was the mp4parser https://github.com/sannies/mp4parser, thank you very much!