andy vt's tools & blog

auldar:

Ahh.. Having just found this thread on the xbmc forums:

 

http://www.xboxmediacenter.com/forum/showthread.php?t=34&page=26&highlight=dvrms

 

I see John Donaghy who you mentioned was the one who did much of the dvrms work is creating new mplayer.dll's for testing on xbmc.. Nice :)

John has been working with the mplayer developers as well to integrate all changes into the SVN.  The mplayer developers are VERY picky about the code, so much of it has been re-written to please them.  Once they accept the changes any new compile of MPlayer or Mencoder will include dvr-ms support.

Arkay:

Hi Jelwood,

I've found some time to run some tests.  Got a recording from each of our FTA digital stations here. 1xSD and 1xHD for each.  Australia is a little weird in that some channels broadcast AC3 audio and others PCM.

On the tests that I've done the only conversions that worked (with playable audio) were the PCM recordings.  It doesn't appear that the demuxer is handling the AC3.  Interestingly DVRMSToolbox and directshow has always suffered the same fate with these recordings.

In all but 1 case though the video converted nicely.  Any AC3 audio resulted in pops and clicks irrespective of -oac or conversion to mp3. 

I was going to supply logs etc but I think it would be much simpler if I can supply you with some test clips.

I have 10 sample files here (1 each of SD/HD for 5 channels), 30 seconds each totaling about 320MB.  I'd be happy to upload them somewhere or ftp them to you (if you have a server).  Please let me know.

Congrats on the work so far.  I love to see MS proprietary codecs made usable by opensource :)

Cheers,

Arkay. 

Can you upload them to megaupload.com?  It's a service that works pretty well for this because you won't have to register.  The clips just have to be smaller than 500mb.

Thanks for your help in testing!

I've been away from the project for a few days. What is the latest with EDL support? I think I have the latest mencoder update but I can't get edl to work. I still don't think I'm calling it correctly.

If someone has it working, are you using Comskip or ShowAnalyzer?  I do have a license for ShowAnalyzer. Do I somehow need to create the EDL file first or does it do it on the fly?

Jelwood,

 

Files are uploaded (all Vista recordings BTW).  You can grab them here:

 

http://www.megaupload.com/?d=EXUF5LOA

 

Be interested to see how you go with the AC3 files.

 

Cheers,

 

Arkay. 

Arkay:

Jelwood,

 

Files are uploaded (all Vista recordings BTW).  You can grab them here:

 

http://www.megaupload.com/?d=EXUF5LOA

 

Be interested to see how you go with the AC3 files.

 

Cheers,

 

Arkay. 

I have some good news and bad news about these clips...  The good news is I think I can fix this.  I only had time to look at one clip (more on why below), but the clip I looked at had AC3 audio, but the dvr-ms header file said it was MP2...Oops.  For a quick test, I ignored the header file and forced it to AC3 and the clip converts just fine.  So, if we can identify the codec correctly, these clips won't be a problem.  Now I just have to "enhance" the audio detection.  Until now we have been relying on what the header file tells us.  I just have to figure out another way to detect the audio.  I won't have time to work on this until next week.  It's my anniversary this weekend and I'm taking my wife out of town, with no computers - we'll see if I can survive :)

Jelwood,

That's fantastic news!  It makes perfect sense too as I spent quite a bit of time stuffing about with directshow graphs trying to make it work.  Every filter I tried identified the audio as mp2 and it was only with a combination of a demuxer and re-muxing back the original dvrms video into an mpeg that I ever got it to work and I could never repeat it after the first time I got it to work.  It was a long time ago and I can't remember the exact procedure but it required two graphs and double processing so I gave up.  I always new MS had done something dodgy in the code in order to support ac3 recordings.  I'm absolutely wrapped that you have identified it! :)

I don't know if you'd be interested but I would love to know the difference between what the ac3 header should look like vs what the files actually contain.  It would then be possible to also write a patcher to patch the recordings themselves so that other conversion tools (even directshow ones), could work with the files.  One step at a time though.  Interestingly the only other people that ever worked this out were the videoredo authors and funnily enough they never shared the information!
 

 Anyway, enjoy your anniversary! and don't stress, there's many more important things than PC's and dvrms files ;)

 

Cheers,

Arkay

 

http://www.megaupload.com/?d=CA4TJQHF

Above is the latest version of Mencoder (Version 5).  This version doesn’t have any improvement in reading the dvr-ms format (compared to version 3).  However, I have done some work on the items below and I wanted to post it before I tore everything apart working on improving AC3 detection.

New things in this version are:

1)     I removed the dependency on cygwin1.dll (this version actually works).

2)     I worked on thread support again (this is why there is a requirement for pthreadGC2.dll).  I can’t get either ffmpeg or Mencoder to use 100% of a dual core system for Xvid encoding (it will use 100% for H.264 encoding).  However, I did update the Xvid library to the latest SVN which now supports SMP (multi-processor systems).  Please try it out and let me know if you get any improvement in encoding times.  There are two ways to access this library.

      The option below will call the xvid library directly

      -ovc xvid -xvidencopts fixed_quant=4:threads=2

      The second option is to have ffmpeg call the xvid library

      -ovc lavc  -lavcopts vcodec=xvid:vqscale=4:threads=2

3)     This version supports both H.264 and AAC.  I don’t own an iPod, so I have no idea if this will play in that device.  However, I did test the playback on my PC and I know it does encode in H.264 and AAC.  I just don’t know all the “tweaks” necessary for the iPod hardware.

Here are the command lines I’ve used for testing.  Notice I’m using vcodec=mpeg4 for Xvid because this gave me the best speed on my system.

Xvid/MP3:

mencoder -lavdopts threads=2 "input.dvr-ms" -vf expand=:::::4/3,scale=320:240 -ofps 29.97 -ovc lavc -lavcopts vcodec=mpeg4:vqscale=4:threads=2 -ffourcc XVID -oac mp3lame -lameopts abr:br=128 -o "output.avi"

H.264/AAC:

mencoder -lavdopts threads=2 "input.dvr-ms" -vf expand=:::::4/3,scale=320:240 -ofps 29.97 -ovc x264 -x264encopts crf=4:threads=2 -oac faac -faacopts br=128 -o "output.avi"

Arkay:

Jelwood,

That's fantastic news!  It makes perfect sense too as I spent quite a bit of time stuffing about with directshow graphs trying to make it work.  Every filter I tried identified the audio as mp2 and it was only with a combination of a demuxer and re-muxing back the original dvrms video into an mpeg that I ever got it to work and I could never repeat it after the first time I got it to work.  It was a long time ago and I can't remember the exact procedure but it required two graphs and double processing so I gave up.  I always new MS had done something dodgy in the code in order to support ac3 recordings.  I'm absolutely wrapped that you have identified it! :)

I don't know if you'd be interested but I would love to know the difference between what the ac3 header should look like vs what the files actually contain.  It would then be possible to also write a patcher to patch the recordings themselves so that other conversion tools (even directshow ones), could work with the files.  One step at a time though.  Interestingly the only other people that ever worked this out were the videoredo authors and funnily enough they never shared the information!
 

 Anyway, enjoy your anniversary! and don't stress, there's many more important things than PC's and dvrms files ;)

 

Cheers,

Arkay

 

http://www.megaupload.com/?d=FQ4SL888

Above is version 5a.  Inside this zip file you will find the normal Mencoder.  However, I also included a version that forces the audio to AC3 (Mencoder-ac3.exe) just in case you would like to try it.

One other thing you could do to help...  I think we can key off of the Metadata tag "WM/WMRVDTVContent".  In the samples I have this tag is true if the audio is AC3 and false if the audio is MP2.  I know this isn't the greatest solution, but right now it's one option (John thinks he has a much cleaner solution).  Can you look at your samples and tell me if this holds up for your samples?  Just in case John’s solution doesn’t work I’d like to have a backup plan.

Edit:

Never mind…  John’s theory about reading the video stream and detecting whether it’s MPG2 or AC3 will work (the prototype worked great).  Now the code just has to be integrated into Mencoder.

Never mind…  John’s theory about reading the video stream and detecting whether it’s MPG2 or AC3 will work (the prototype worked great).  Now the code just has to be integrated into Mencoder.

Never mind…  John’s theory about reading the video stream and detecting whether it’s MPG2 or AC3 will work (the prototype worked great).  Now the code just has to be integrated into Mencoder.

auldar:

Any idea's how you could use dvr-ms support in mencoder to dump out to mpeg for use with something like Auto-GK (I know I know.. just asking :P).

Would it re-encode, or just strip off the encapsulation?

I suspect this functionality is in the base DVRMSToolbox, but being a vista user, I don't have access to a working version.
 

http://www.megaupload.com/?d=J1LA91LE

I’ve done some testing and ffmpeg is a lot better at writing the mpg container.  So, above is a link to a version of ffmpeg that you can test with.  I haven’t figured out the best command line to use.  However, I did have some success with the command line below.  I still have issues with some HD shows with this command line, however, the video and audio are there.  There is something in the container header (maybe the fourcc value) that Windows Media Player just doesn’t like because I can play the files just fine using MPlayer (I included MPlayer in case anyone wants to test with it).  Hopefully, someone can figure out the best command line for us.

"ffmpeg" -benchmark -y -i "input.dvr-ms" -vcodec copy -acodec copy –target ntsc-svcd –o "output.mpg"

 

P.S.  I wouldn’t try using this version of ffmpeg for anything but reading the dvr-ms container and putting the video and audio into a mpg container.  I don’t have all the dvr-ms format information ported to ffmpeg yet (I just ported enough to make this work).

 

Jelwood,

This is fantastic! Nice work :)  I've just spent some time playing with the ac3 version.

Ironically I'm doing all this in linux using wine to run the windows version of mencoder! But anyway, I can confirm that the ac3 files are converting nicely.  I need to do some more tests in a real windows environment as I've seen a couple of glitches towards the end of files (audio stops quiet a bit before video does). etc.  I really don't know if they are related to running this in a linux environment.  Just wanted to give you some instant feedback.  Tonight (hopefully), I can perform some full tests on windows and get back to you.  I'll also check out the tags and see if they are consistent with all recordings.

Many thanks for working on this, stay tuned for some more infomation.

Cheers,

Arkay.

 

jelwood:

auldar:

Any idea's how you could use dvr-ms support in mencoder to dump out to mpeg for use with something like Auto-GK (I know I know.. just asking :P).

Would it re-encode, or just strip off the encapsulation?

I suspect this functionality is in the base DVRMSToolbox, but being a vista user, I don't have access to a working version.
 

http://www.megaupload.com/?d=J1LA91LE

I’ve done some testing and ffmpeg is a lot better at writing the mpg container.  So, above is a link to a version of ffmpeg that you can test with.  I haven’t figured out the best command line to use.  However, I did have some success with the command line below.  I still have issues with some HD shows with this command line, however, the video and audio are there.  There is something in the container header (maybe the fourcc value) that Windows Media Player just doesn’t like because I can play the files just fine using MPlayer (I included MPlayer in case anyone wants to test with it).  Hopefully, someone can figure out the best command line for us.

"ffmpeg" -benchmark -y -i "input.dvr-ms" -vcodec copy -acodec copy –target ntsc-svcd –o "output.mpg"

P.S.  I wouldn’t try using this version of ffmpeg for anything but reading the dvr-ms container and putting the video and audio into a mpg container.  I don’t have all the dvr-ms format information ported to ffmpeg yet (I just ported enough to make this work).

I figured it out...  Below is the command line that works.

"ffmpeg" -i "input.dvr-ms" -vcodec copy -acodec copy -f dvd "output.mpg"

Using version 5a and it's looking great! I've got a bit more testing to do. Then I'll post my scripts. Looks like EDL is working for me too.

Slightly off topic but posted here because it has to do with the original file that I will be working with. Quick question for anyone using Vista media center (may apply to XP too, not sure) and an HD card....
When I used Vista media center to save from an HD channel (true HD content) the files get saved as dvr-ms (as expected) and I right click to see the properties and I see the dimensions as 704x480. I wonder if it is because I'm using media center as a computer hooked up to a 4:3 monitor and have MCE set up for that.Or maybe it's my tuner?
Anyone else notice this? Or are your original dvr-ms files getting saved at the full HD size of 1920x1080 or 1280 x 720 depending on the source?

-Scott 

kingtone:

Using version 5a and it's looking great! I've got a bit more testing to do. Then I'll post my scripts. Looks like EDL is working for me too.

Slightly off topic but posted here because it has to do with the original file that I will be working with. Quick question for anyone using Vista media center (may apply to XP too, not sure) and an HD card....
When I used Vista media center to save from an HD channel (true HD content) the files get saved as dvr-ms (as expected) and I right click to see the properties and I see the dimensions as 704x480. I wonder if it is because I'm using media center as a computer hooked up to a 4:3 monitor and have MCE set up for that.Or maybe it's my tuner?
Anyone else notice this? Or are your original dvr-ms files getting saved at the full HD size of 1920x1080 or 1280 x 720 depending on the source?

-Scott 

I noticed this as well...  Windows Explorer must be pulling this information from the dvr-ms header.  When we did the work in Mencoder to read the dvr-ms format we had to ignore this value because we noticed it's incorrect (along with a lot of other things that are incorrect in the header).

jelwood:

kingtone:

Using version 5a and it's looking great! I've got a bit more testing to do. Then I'll post my scripts. Looks like EDL is working for me too.

Slightly off topic but posted here because it has to do with the original file that I will be working with. Quick question for anyone using Vista media center (may apply to XP too, not sure) and an HD card....
When I used Vista media center to save from an HD channel (true HD content) the files get saved as dvr-ms (as expected) and I right click to see the properties and I see the dimensions as 704x480. I wonder if it is because I'm using media center as a computer hooked up to a 4:3 monitor and have MCE set up for that.Or maybe it's my tuner?
Anyone else notice this? Or are your original dvr-ms files getting saved at the full HD size of 1920x1080 or 1280 x 720 depending on the source?

-Scott 

I noticed this as well...  Windows Explorer must be pulling this information from the dvr-ms header.  When we did the work in Mencoder to read the dvr-ms format we had to ignore this value because we noticed it's incorrect (along with a lot of other things that are incorrect in the header).

Interesting. So it really is HD resolution, just not coded right in the header? 

kingtone:

jelwood:

kingtone:

Using version 5a and it's looking great! I've got a bit more testing to do. Then I'll post my scripts. Looks like EDL is working for me too.

Slightly off topic but posted here because it has to do with the original file that I will be working with. Quick question for anyone using Vista media center (may apply to XP too, not sure) and an HD card....
When I used Vista media center to save from an HD channel (true HD content) the files get saved as dvr-ms (as expected) and I right click to see the properties and I see the dimensions as 704x480. I wonder if it is because I'm using media center as a computer hooked up to a 4:3 monitor and have MCE set up for that.Or maybe it's my tuner?
Anyone else notice this? Or are your original dvr-ms files getting saved at the full HD size of 1920x1080 or 1280 x 720 depending on the source?

-Scott 

I noticed this as well...  Windows Explorer must be pulling this information from the dvr-ms header.  When we did the work in Mencoder to read the dvr-ms format we had to ignore this value because we noticed it's incorrect (along with a lot of other things that are incorrect in the header).

Interesting. So it really is HD resolution, just not coded right in the header? 

Correct...  Look Mencoder and it should tell you the correct resolution.