Getting Started with HTTP Dynamic Streaming

As described in the first post in this series: What is Dynamic Streaming?, HTTP Dynamic Streaming (HDS) allows for the delivery of streaming content over the HTTP protocol. Streaming video over HTTP requires some knowledge and preparation. The following is a rundown and description for each of the components involved with HTTP streaming.

The files: The F4F and F4M file formats

The F4F file format specification (http://www.adobe.com/devnet/f4v.html) describes how to divide media content into segments and fragments. These segments and fragments are what make ‘streaming’ content over HTTP possible. Basically, the file is broken up into multiple pieces and parts when it is prepared for for HTTP streaming.

A sample segment and its fragments
A sample segment and its fragments

These way the segments and each segments fragments are created is based on multiple factors such as the length of the video content, the the number of keyframes and the length specified for each segment. The result of packaging your video content for HDS delivery is a set of files that look similar to the following for a simple, single bit-rate file:

HTTP Dynamic Streaming Packaged Files for a short video file
The set of files created by the f4fpackager for a short video clip.

From the image above:

  • sample.f4m: This is an XML file created by the f4fpackager that contains the information necessary for a client player to playback the video file. There will be only one of these for each source video file.
  • sampleSeg1.f4f: This segment file contains the fragments that the client player will request for playback. There can be multiple of these files for each source video file.
  • sampleSeg1.f4x: This is an index file that contains specific information about the fragments inside the segment files. There will be one of these types of files for each segment file. The HTTP Origin Module uses the data in this file to determine the fragment to send back to the client after a request.

Each F4F file is a single segment and the segment can contain multiple fragments. So, if you inspect the HTTP requests as a media item is being played back you will see files being requested that map to a fragment within a segment. For example:

Segments and fragments being requested over HTTP
Segments and fragments being requested for HDS video

This request specifies to the HTTP Origin Module what segment’s fragment to pull out of the F4F file and deliver to the client for playback. These requests are based on where the time code of where the media item is during playback. So if you scrub to later in the video, the Requests might look something like the following:

Scrubbing HTTP Dynamic Streaming Video Content
Segments and fragment requests while scrubbing HDS video content

This allows the client to request any piece of the video content and start playing back almost immediately. You can control how the files are segmented and fragmented when preparing content with the f4fpackager. The basic concept is to balance the size of the fragment being delivered and the number of HTTP requests that a client needs to make for playback. Please note, the fragments may not be sequential, so you cannot rely on this when requesting content. The the fragments are based on the settings passed into the packager and can skip fragment numbers. So , you can expect to see fragment sequences like the following (no scrubbing involved):

  1. sampleSeg1-Frag1
  2. sampleSeg1-Frag2
  3. sampleSeg1-Frag3
  4. sampleSeg1-Frag5
  5. sampleSeg1-Frag6

This doesn’t mean that the file is incomplete, it is just how the fragments were created by the packager.

The Details

The F4M File Format

The F4M or Flash Media Manifest file format contains the information about the package of files created when video content is packaged for HDS. The information included in the manifest file can include some or all of the following:

  • Media location
  • Media type
  • Media bootstrap
  • Multi-bitrate (MBR) availability
  • Codecs
  • Resolutions
  • Digital Rights Management (DRM) authentication
  • DVR information
  • moov atom, metadata block and XMP metadata block

When playing back HDS video content, the F4M file is expected to be loaded as the ‘media file’. The client is responsible for inspecting the data included in the F4M file to authenticate (if DRM authentication is required), determine if MBR content is available and select the best MBR version of the content and then request the media from the server.

Sample of a simple F4M file with a single media item:

[xml][/xml]
Installing and configuring the Apache HTTP Origin Module: http://help.adobe.com/en_US/HTTPStreaming/1.0/Using/WS7b362c044b7dd076-735e76121260080a90e-8000.html

39 Replies to “Getting Started with HTTP Dynamic Streaming”

        1. F4F is based on the same spec as F4V the difference being an F4F file contains fragments (built using the F4V file spec – headers, boxes etc) grouped into segment files (the actual F4F file). The PDF at http://www.adobe.com/devnet/f4v.html describes the headers and boxes that you deal with when creating or playing back F4F fragments. Look at "Annex C. HTTP Streaming: File Structure" (pg. 63).

          Creating F4F files is a complex process of breaking up the source video file into the F4F segments and their contained fragments. I don't have a deep knowledge of the process to manually create the F4F files. This is where I rely on the f4fpackager tool to create the F4F files and the OSFM framework (specifically the org.osmf.net.httpstreaming package) for dealing with F4F during playback and for some of the tools that I've built.

          Are you trying to accomplish something specific or just getting a better understanding of the format?

    1. Absolutely. Let me know if there is anything I can do to make it easier for you. If you wouldn’t mind giving me a little attribution, that’d be appreciated.

  1. Two questions:

    1. What is the state of browser and mobile (Android) support for segmented streaming F4V?

    2. Do you have FFmpeg/x264 profiles for encoding segmented streaming F4v?

    Thank You,

    QD

    1. Ludodos – thanks for the info on #1.
      QD – #2: I don't have any profiles for encoding – sorry. But I'd be happy to ask some of the resources I have to get some info on it.

    1. Sounds like another feature I'll have to add. For now you would need to run each file bitrate through with its own F4M.

  2. Hi John,

    Thanks a lot for the detailed article. It really helps in understanding HDS in a simple way. I have a follow up question.

    In your article, you mention " NOTE: OSMF provides the code to handle playback of fragments as well as determining which segment/fragment pair to request based on the current time of playback. This code exists in the package org.osmf.net.httpstreaming." ..

    Now my question. Does this mean that anyone using a OSMF client, will not need the HTTP Origin Module since the client is already helping out in picking the right fragment. So an Apache/IIS/AmazonS3 server + OSMF client should suffice to deliver HDS content?

    1. Hi Rohonesh – I'm happy to hear the article has been helpful.

      To answer your question, OSMF provides the logic to parse the F4V fragments into FLV byte data that can be fed to a NetStream using the appendBytes() method. So, the HTTP Origin Module is required to provide the client with the correct Fragment for the current time. The situation where the HTTP Origin Module is not necessary is when the fragments have been extracted into explicit files. Then these fragment files can be served from any server and the HTTP Origin Module is not required.

  3. is there a way to turn these fragments of f4f file into a flv. or mp4. file? I’ve got a paid account in a website and they don’t let download videos anymore 🙁

  4. Hi,

    Is it possible to reduce the number of requests from clients to server in live f4f streaming? Live streaming means that I don't have any f4f files, I get video from satellite in real time, transcode it in real time, create fragments and send them to client by http. Max allowable delay is 5 sec. So I must create fragments with max duration of 5 secs and every 5 seconds every client will ask server for next fragment. Too much requests. But is it possible to send next fragment to client without any request from him, in the same opened http connection?

    1. The only real way to reduce the number of requests while using HDS is to adjust the size of the fragments and segments when you package the content. If you have to create fragments that are of a specific size then, this isn't an options. Have you thought about using a CDN to help with the number of requests. You could also look into other protocols like RTMP or even some sort of multi-cast solution to minimize the amount of requests hitting your server.

      1. To create fragments all the same size – its good idea, I never thought about it. As I use own developed codec, its possible to do. Or maybe CDN. RTMP – I thought about it, scrolled specification, but as I can observe, video in Internet gradually go away from RTMP to mp4 (f4f). And as I never worked with rtmp, study it now – don't know how it will be useful. Especially if MPEG DASH/HTML5 will be accepted as a standard.
        Thanks a lot!

  5. Hi John,

    Is it possible to include the content of manifest file (.f4m) directly into html code of web page? In this case flash player will not request f4m file from server, it will get it directly from html. And if it is possible, are there any examples available of how to do it?

    1. Hi Harry –
      I haven't ever thought of that. Conceptually it seems like it would be plausible – the only issue is that OSMF uses an F4MLoader Element (extended from Document Element) to load the XML using a URLLoader so there may be an issue with that and some work under the hood of the player to make it realize that it needs to load the XML in another way.

      P.S. Sorry for the delayed response. My blog has been neglected lately.

      1. My first thought was to understand how can I develop such a player. But then I thought – does flash player always use the same mechanism of loading xml files, no matter what kind of xml it needs? If answer is "yes" then, I think, it is a great security problem – in this case it is possible to implement player that will not load crossdomain.xml. Very interesting question.

  6. Hi,
    Are there a way to get a time stamp of when the download of the current fragment was completed? On ActionScript in flash player or on JavaScript.

    1. @Harry_m As far as I know there isn’t anything currently set up in the OSMF api for this. But, I’m sure it could be tracked down in the org.osmf.net.httpstreaming package. I’m not sure what class handles the downloading off hand though.

    1. Thank you for the quick response.
      Also, I downloaded adobe media server and using f4fpackger from the tool folder i generated some sample segment files ( sampleSeg1.f4f, sample.f4m, sample.f4x) of a flv video. i would like to know if i can analyse these files? if yes, which tool shall i use?
      Regards.

  7. Hi John,

    Thanks for your article and the replay to the questions asked above. I am working on multiple http streaming technlogies like http live streaming, dash, smooth streaming, Now I wanted to work for the HDS technology.
    1) Do I need to pay any royalty to use the HDS spec from Adobe?
    2) Since I cant use OSMP for my player, as I am using ffmpeg framework. Does fmpeg support HDS also? Can I get a HDS parser open source library like libdash.so which parses the dash manifest file.
    3) Can you give me the complete link of the HDS spec, so that I can go through that spec.

    I can see so many people interacting with John, if you guys have any info about my questions, please reply back to me.
    Thanks and Regards,
    Sunil Deshpande

    1. Hey Sunil, I’m glad to be helping out!
      To answer your questions:

      1. You’d need to make sure to check it out yourself, but I don’t think there are any licensing to use HDS.
      2. FFMPEG doesn’t package HDS content, but you could use the f4fpackager (a tool bundled with Adobe Media Server) to package the content. Since you’re using FFMPEG already, it should be trivial to add in another cmd line tool.
      3. HDS is a combination of files and formats:
        1. F4M Flash Media Manifest File Format
          v2.0: http://sourceforge.net/apps/mediawiki/osmf.adobe/index.php?title=Flash_Media_Manifest_(F4M)_File_Format
          v1.0: http://sourceforge.net/apps/mediawiki/osmf.adobe/index.php?title=Flash_Media_Manifest_(F4M)_File_Format_obsolete
        2. F4V’s video file format: http://www.adobe.com/devnet/f4v.html
    1. The bootstrap data contains a fragment run table that give you the names of the fragments to download (Seg1Frag1, Seg1Frag2, Seg1Frag3 – they aren’t always sequential). Then you can download each fragment. Once you have all of the fragments, you need to reassemble the video data based on the F4V file spec & you can play it back.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.