I have done a lot of work with Windows Media files in the past where it was a requirement to capture, embed, transmit, and display the actual timecode that was generated on a broadcast tape for the purpose of logging a file remotely.
This use case is common in the broadcast world where it is very cost effective to have a browser-based proxy viewer instead of $20k DigiBeta deck to do simple shot logging and commenting.
In the Windows Media 9 days we (MSFT) provided tools in the Windows Media 9 encoder, and in the Windows Format SDK, to allow for this type of workflow. The Data Unit Extension (DUE) in the Format SDK was provided for this purpose and allowed the developer or encoder to inject the SMPTE timecode into the per-frame DUE by populating the WMT TIMECODE EXTENSION DATA structure with the appropriate range, timecode and custom flags that you wanted. This was very useful and was implemented by a number of encoder manufacturers. The nice part about this was that it also provided support for time code breaks on a tape, which is a common scenario with older tape based cameras (new cameras just create a seperate MXF digital file each time I stop and start the camera, so it is not a big issue any more).
So, in our past we developed all of this wonderful code to embed SMPTE timecode into our files and a way to read it out in Windows applications that used the Format SDK and even support in the Windows Media Player OCX browser control to read and navigate this timecode. Then along comes Silverlight(tm) 1 and 2.
In Silverlight we now have the MediaElement. It does not do all of the cool stuff that the old WMP.ocx allowed with SMPTE timecode, but I have provided a workaround to that in our IMM solution. That workaround has now graduated to the Expression Encoder 2.0 SP1 and is available (in source!) as part of the Silverlight 2 Media Player template.
You can easily convert the TimeSpan object that is reported back from Silverlight’s MediaElement to SMPTE 12M timecode using this Timecode class that I created. You can also use it to seek a specific SMPTE timecode, build a Silverlight Rough Cut Editor, or build a timecode calculator application.
If you have Expression Encoder 2.0 SP1 installed go to this location to find the Timecode class.
C:\Program Files (x86)\Microsoft Expression\Encoder 2\Templates\en\SL2Standard\Source\MediaPlayer\TimeCode.cs
This is a struct that works just like the TimeSpan in .NET. You pass it the TotalSeconds from the CurrentPosition timespan of the MediaElement and the SMPTE 12M framerate that you want calculated (say for example SMPTE 29.97 Drop Frame), and the Struct will give you back a valid SMPTE timecode string.
You can also use it in reverse. Pass it a time and framerate that you want to go to, and get back the Absolute time in seconds, then set the current position to that.
As an example of using the Timecode.cs file, I have created a very simple Silverlight 2.0 player application in Blend 2.0 SP1.
It has a single MediaElement, a Slider control, a Play and Pause Button, and a timecode display.
For this example I am using a file that I created in Sony Vegas that is just a window burn of SMPTE timecode running at 29.97 Drop Frame. Drop frame is the hardest algorithm to do, so let’s stick with that one for now.
I uploaded the file to Silverlight Streaming so that you can use it also in your testing.
http://silverlight.services.live.com/1535/SMPTE_NTSC_Drop/video.wmv
In my XAML page’s code behind, I added a DispatcherTimer to fire an event to update the Timecode display every 500ms.
public partial class Page : UserControl { private readonly DispatcherTimer timer; private bool isSliderDragging = false; public Page() { InitializeComponent(); this.Player.AutoPlay = true; this.timer = new DispatcherTimer { Interval = new TimeSpan(500) }; this.timer.Tick += new EventHandler(this.Timer_Tick); this.timer.Start(); }
When my Timer ticks I update the Timecode display by grabbing the value of the TimeSpan in the Player.CurrentPosition and then pass it to the Timecode struct with the proper SMPTE 12M framerate that I want calculated. The struct then returns back a formatted SMPTE 12M timecode string.
private void UpdateTimecode() { var tc = SmpteTimecode.TimeCode.FromTimeSpan(Player.Position, SmpteFrameRate.Smpte2997Drop); TimeCode.Text = tc.ToString(); }
One thing to note is that I actually added a new function to the Timecode.cs file in this case to directly pass a TimeSpan in. This was not in the original code that shipped in SP1. It just passes the value of TotalSeconds to the constructor of TimeCode that takes a double in seconds.
public static TimeCode FromTimeSpan(TimeSpan value, SmpteFrameRate rate) { return new TimeCode(value.TotalSeconds, rate); }
The result looks like this. You can drag the slider around in this file and see that the reported SMPTE 12M NTSC drop frame timecode in green perfectly matches the window burn inside of the video file (the number at the top is framecount).
Unfortunately my hosted WordPress is not allowing me to embed Silverlight Streaming applications right now, so you can play with it live here.
(Note: if you are on a slow connection you should wait for the whole file to download or the seeking won’t be accurate).
Now, the obvious limitation is that we can’t easily deal with timecode breaks. One workaround for this that I have used is to embed the timecode breaks into the VC-1 file as Markers. In the marker you can specify that this is a break, and then in a custom Silverlight player when you hit that break you will easily be able to update the timecode that is being displayed by using the Timecode struct class as a Timecode “calculator”. Store the offset time in a Timecode and just add it to the current position and you will have a properly offset timecode to display!
Enjoy. It took me a long time to put all of those math operator precedence parentheses into this file so please send me email if you end up using this in your own projects. I would love to hear.
Also, you may have already considered this, but this class alone is a great base for a Silverlight based Rough Cut Editor solution. Just build a timeline control and playback engine based on the Timecode class.

29 comments
Comments feed for this article
February 22, 2009 at 11:45 am
Oded
Thank you for the great post.
Is there a way to handle video files that their timecode does not start from 00:00:00.00?
Many files injested from tapes contains TimeCode that is relative to some other marker, and not the “absolute zero”…
Thanks again,
Oded
September 3, 2009 at 2:09 am
John
Hi Guy’s, be careful with the Timecode class: I found that it could not count the last 25 frames of the day. I had to adjust it’s internal max value to allow for 23:59:59:24 (25 frames per second).
John.
February 22, 2009 at 5:24 pm
John Deutscher
One method that I have used successfully is to store the Start Timecode for the asset as metadata in a database (DAM system or custom database).
Then in your custom Silverlight player you can load the metadata about the asset and set the two most important parameters – the timecode to start counting from, and the framerate to assume.
// set the starting timecode – get this from database, or metadata on the server.
TimeCode startTimecode = new TimeCode(“00:58:00:00″, SmpteFrameRate.2997NonDrop);
TimeCode currentTime = new TimeCode(Player.CurrentPosition, SmpFrameRate.2997NonDrop);
// Use the timecode struct like a calculator. This also works with mixed rates.
TimeCode displayCode = startTimecode + currentTime;
// display the timecode to the user interface.
DisplayTimeControl.Text = displayCode.ToString();
February 23, 2009 at 5:33 am
Oded
This is indeed a good solution, but extracting the SOM (or the TimeCode offset) is not a simple task at all…
How do you suggest extracting this information using a windows service that needs to “scan” a huge storage containing many Giga-Bytes of video files?
I know that the windows media player activex object can extract SOM’s, but it can’t be hosted inside a windows service (due to the lack of message pump caused by the lack of GUI)
And running inside a windows application is not an option (and unacceptable by our clients)
Thanks again for the priceless info.
February 23, 2009 at 8:20 am
John Deutscher
It depends on the type of files that you have stored. If the files contain some sort of metadata header that contains the SOM code, then this sould be very doable.
I currently have my own Windows Service that hosts a REST and SOAP endpoint that allows me to query for the metadata inside the file header from a Silverlight application.
The service uses the Windows Media Format SDK to read the Timecode ranges from the file (note that this only works with WMV files – but the service can be pluggable for other known metadata headers). The format SDK will return the SOM and the list of “ranges” in the WMV file (same API that the Media Player uses). This solves the other issue – timecode breaks… as long as your encoder is able to place the proper ranges into the WMV file on encode. It all depends on what is currently in the “huge store” that you already have.
February 23, 2009 at 12:18 pm
Oded
I would be very happy if you could write a short post on how to extract those metadata headers using the Windows Media Format SDK from within a windows service…
I do handle WMV files as a video format.
Anyhow, you’ve got me thinking, and I’ll do my own research on the subject. Many thanks, and yet again – it is a great blog post!
March 17, 2009 at 4:42 pm
Tony Lockard
John
Great article – we’ve been swapping emails about this seems like for years glad to finally see it set out in a clear way.
Will MS be pushing out any of the other cool stuff you developed as part of the IMM team to Codeplex or into the Silverlight control set (2.0/3.0)?
The video timeline, mediabins, mark-in/out, etc silverlight stuff you created in the IMM RCE is very cool, have seen nothing like that available from commercial tools or on the open-source silverlight sites…
Best,
Tony
March 18, 2009 at 5:09 am
John Nicholl
Hi I loved your article. Thanks.
I had one small issue: I installed Encoder 2 from the link you provided, However the file:
“C:\Program Files (x86)\Microsoft Expression\Encoder 2\Templates\en\SL2Standard\Source\MediaPlayer\TimeCode.cs”
does not exist and neither does the directory SL2Standard. I would really like to make use of TimeCode.cs any chance you could email me it please?
March 18, 2009 at 5:35 am
John Nicholl
found it here: http://slextensions.codeplex.com/SourceControl/changeset/view/22164#293821
March 18, 2009 at 9:19 am
John Deutscher
You need to install the Sp1 for Expression Encoder to get the file.
August 14, 2009 at 6:02 am
Dan
John:
Great stuff you’re doing here – I too have been (am) fighting the Timecode bits w/ SilverLight. While your solution does seem doable from the perspective of stored video (on-demand), I don’t think it can help with a live scenario.
Consider the desire to mark points of interest in a running live video. The only thing we have is Timecode (ie, no offset available for live media). And with no way to extract the value from SilverLight, we’re left with using WMP to get this done. Or perhaps some kloodgy synchronized “clock” construct, if one has control of both the live encoder and the viewing experience, but this is inherently error-prone and messy (especially in browser-based applications).
Any ideas here? Thanks again for the great work.
–dan
August 25, 2009 at 9:17 am
John Deutscher
Dan,
The live scenario is obviously still a tricky challenge. My solution does not currently address the live scenario, where it is still beneficial to have some sort of User Data Extension or embedded VITC like data block in the packets. I am hoping that the IIS Smooth Streaming team will eventually provide that level of support since each packet that is received by the IIS Smooth streaming parser is could potentially contain the SMPTE time code stamps from the hardware encoder directly in the header metadata. Keep an eye on what the IIS Smooth Streaming team and SDK is doing and be sure to contribute ideas to that team to improve the platform.
August 26, 2009 at 7:22 am
Lewis Kirkaldie
Having quickly just written for the millionth time a simple converter for non-drop timecodes I stumbled into this post! I *knew* I’d seen a decent class for this before – just couldn’t remember where. I’ll plug this code in instead – it is for use in an Azure-based demo which will be at IBC (possibly even on the Microsoft stand!).
However, unfortunately it did not solve the problem I wanted to solve – is there an in-client mechanism to read the ‘NaturalFrameRate’ from a video clip *within* the MediaElement? I don’t have the luxury of a DAM system behind the clips served to my player – it’s all user-upload stuff.
I want to hook this up to the player using an IValueConverter – which works, but I have to pick a framerate out of thin air…
See you at IBC!
October 13, 2009 at 5:00 am
Sergey Novikov
Nice feature, but I have doubts about accuracy.
I see a problem here in your example. Here is what is wrong: if we press play and pause several times – and sometimes time code does not match what i see in video (+- one frame).
John, do you know why is it happen and how to solve this problem? Broadcasters are very sensitive to all related to time code, so this could be big deal for them.
October 14, 2009 at 9:08 am
John Deutscher
Sergey,
The play/pause issue is actually just a refresh issue with the simple player that I put together. In that player I did not add a refresh to the displayed timecode after the play/pause event so it sometimes can be off by a frame. If you just drag the slider bar around though you should see accurate results.
The version that shipped in the Expression Encoder has a known off by 1 frame rounding issues with 23.98 content ( I have a fix for that if you are interested) but my testing showed that it was accurate for NTSC and PAL framerates.
October 14, 2009 at 8:06 pm
Sergey Novikov
Thanks for answer, John.
Yes. I do interested in fix, because in my workflow accuracy is a big issue to client.
October 23, 2009 at 6:51 am
Sergey Novikov
Hi John.
We’ve been looking for a while on timecode problem and found that for 25fps correct AbsoluteTimeToFrames function will ber:
Convert.ToInt64(Math.Floor(25 * absoluteTime));
instead of just Convert.ToInt64(25 * absoluteTime);
But now we are facting another strange problem we also found in all Silverlight players including official microsoft sites:
If we try to set mediaElement.Position = mediaElement.NaturalDuration.TimeSpan or position of aproximatly last 1-2% of video mediaElement will set incorect postition. For example for 1 hour video it will be last 35 seconds.
Still we can play and pause this secons correctly.
Do you know about this problem or how to fix it?
October 15, 2009 at 9:09 am
tony lockard
Great set of posts John. Any updates/changes with SL3/Encoder 3? What about MP4 vs. WMV?
October 24, 2009 at 4:47 am
John
Hi John,
I’ve been profiling my app which uses the Timecode class using Eqatec profiler.
I’m new to profiling and still getting my head round all that is presented. However it looks like the conversion from absolute time ToString() (25fps) takes about 80ms this is 2 frames of time!
I have created a timecode clock using the class, for GUI use, and I am concerned that 40ms timer hitting an operation that takes 80ms could cause me problems. (curiously it does not seem too) Cautiously I reduced my Timer interval to 160ms and display every 4th frame.
Do you have any thoughts on this potential performance issue?
regards
John.
October 24, 2009 at 5:02 pm
John Deutscher
John,
I have not done any profiling of that method myself, but that sounds about right. For most of the apps that I have used the TimeCode class in I have found that 200ms updates is just fine for refreshing the UI text of the timecode as it it ticking along. I’m not sure about your specific use case, but I can’t seem to get my brain to even read the frames as they are ticking along that quickly anyways.
I would not bother trying to update the UI for every frame unless you have some really specific need to be that accurate (although you will likely be more constrained by the speed of the user that is interacting with your app).
October 26, 2009 at 4:48 am
Kalyan chakrvarty
John,
How can i validate the TimeCode in a efficient way, so that i can catch the wrong entry. Can you please help me regarding this issue.
Kalyan
October 26, 2009 at 3:31 pm
John Deutscher
Kalyan,
I’m assuming you mean that you want to validate the entry of some timecode? Usually that should be done at the UI level, but I also do it when parsing strings passed into the timecode class.
In the Timecode.cs file I just used a Regular Expression to validate the format of the timecode and then checked to see if the data was in the expected range.
Regex = “(?\\d{2}):(?\\d{2}):(?\\d{2})(?::|;)(?\\d{2})”
You actually can just split the string on “:” and “;” and validate the ranges on the results. I think each of my functions validates the range of the frames specifically for the specific format (25, 24, etc..).
October 29, 2009 at 2:25 am
Sergey Novikov
Hi, John.
Do you have any thoughts about workaround on 1% ussue with mediaelement, or do you have contact of someone I can ask about it?
October 29, 2009 at 10:51 am
John Deutscher
There are no current workarounds other than to stretch out the length of the file and put in a marker where you want the end to be and to pause / stop when you reach that point. However, this workaround will change other behaviors. For instance, you won’t get MediaEnded events using that technique since you’ve effectively invented you own MediaEnded event that happens before MediaEnded.
Beyond that, the only other workaround that even remotely comes to mind is build a MediaStreamSource and handle all of the samples yourself, but that is a fair amount of work.
November 6, 2009 at 8:05 am
[Silverlight] TimeCode aus .wmv auslesen - Forum Fachinformatiker.de
[...] [...]
January 6, 2010 at 9:03 am
John Deutscher
Gern geschehen. Lassen Sie mich wissen, wenn Sie irgendwelche Fragen zu sehen.
Danke
December 30, 2009 at 8:27 pm
Polling Video – A Viable sub-second alternative? - Jesse Liberty - Silverlight Geek
[...] regard to my intended use of embedded “heartbeat” markers. Mike Loynd (WL) referred me to this fascinating article by John Deutscher (PM for IIS Media). That caused me to experiment with [...]
September 4, 2009 at 1:09 pm
John Deutscher
Thanks John, that is a good catch. That is a bug in the current implementation.
I set the maxValue to 86399 for drop frame, but that will only get you up to 23:59:59:00 at 25 fps. Short by 25 frames.
If you look at the function:
public static double MaxValue
{
get { return 86399; }
}
The actual max values for a 24 hour day are the following, and that MaxValue property would need to take the current framerate into account.
I’ll see if I can get that issue addressed in the next release, but for now you can modify that function to fit your needs if you have a use case for timecode going out to 24hrs.
Rate Max Max allowed TimeCode value would be…
23.98 86486.358291666700000 23:59:59:23
24 86399.958333333300000 23:59:59:23
25 86399.960000000000000 23:59:59:24
29.97 DF 86399.880233333300000 23:59:59:29
29.97 ND 86486.366633333300000 23:59:59:29
30 86399.966666666700000 23:59:59:29
October 24, 2009 at 5:17 pm
John Deutscher
Sergey,
The 1% issue at the end of a video was something I had not seen before in testing. I went back and looked some more at it and asked around and it is apparently a known behavior of the Silverlight MediaElement runtime. Seeking at the last 1% of a file results in the runtime seeking to the lesser of the NaturalDuration *0.99 or 1 second.
There were designed reasons for this I believe.
The issue that you found in the conversion for PAL was also addressed in my latest updates to the Timecode class. I have actually been switching from decimal math to float for some of the functions to fix some precision rounding issues.