Personal View site logo
Wanted some gear but couldn't afford it? It is perfect chance to get it for cheap. Special deals for amazing prices.
If you want to support this project, select one of the options below.
It is going to reversing and project support costs (hosting, etc).
I am spending good amount of time on it and your support allows to improve and expand this site.
contribution size
Official StreamParser topic
  • This is an update to StreamParser to more accurately handle GH2 files. The camera autoselect will detect GH2 files as either GHx files or GH2 files. Some of the file formats for the GH1 and GH2 are the same, so GH2 is only detected when a file format that is uniquely a GH2 format is detected. If you are testing hacked files you might not want to rely on the camera autodetect and select your camera type manually as hacked files often do not conform to what the camera autodetect expects to see.

    Remember to uninstall any old versions of StreamParser first.

    Chris
    StreamParserInstall.zip
    363K
  • 77 Replies sorted by
  • Thanks Chris and Vitaliy.
  • Per Vitaliy's request I'm posting the StreamParser update here. I'll post updates here from now on.

    This version fixed a bug where video bitrates were displayed incorrectly because of a overflow error.

    Chris
    StreamParser Install.zip
    363K
  • would a linux version of streamparser be any easier to write than a mac version?
    i'd like to be able to run something without windows
  • Sorry, I have no plans to produce other versions of StreamParser at this time. Any additional version would be quite a lot of work.

    Chris
  • thx, StreamParser works also for canon hf100 mts files....
  • Per Vitaliy's request I am posting a new version of StreamParser here.

    Several minor improvements and bug fixes.

    The main new feature is the integration of a couple of new tools:

    Under Tools you can now produce a H.264 elementary video stream file from a MTS file. The new file has a suffix of ".264" and is saved in the same folder as the original MTS file. These files are compatible with several editors and analysis programs.

    There is also an option to analyze elementary stream files with h264_parse (integrated) once you have created such a file. h264_parse provides a lot of analysis information such as Quantization Scaling Matrix information, among other things.

    Be sure to uninstall previous versions of StreamParser first (the program name is "GH13 Stream Parser" in the Control Panel.

    Chris
    StreamParserInstall.zip
    396K
  • This is wonderful news.
  • Great job and thank you for all of your effort.

    Does this version display the new 1080p @ 30 FPS video as 1080 30p or does it still say 1080i?
  • I don't know - it depends on how the codec reports it. Try it out and let me know. Is the 30p in a wrapper?
  • No problem. It is in a 1080i wrapper so that is expected. I just thought people will get confused when they still see 1080i for the 25p and 30p modes if they ever install 1.1 Not a big deal though.
  • This is very good news. :-)
    Thanks Chris and VK!
  • Thanks Chris!
  • Comments about the new features in StreamParser: The h264_parse analyzer was not written by me - it's a publicly available, fairly old, H.264 analyzer. It has some limitations - that is, it can't completely parse some of the high bitrate streams all the way through. Someday I will try to fix it - but it will be a while. The elementary stream extraction feature was written by me, and as far as I can tell it does correctly extract the entire stream.

    Most analyzers can process the elementary stream files much faster than MTS files. Also, if you are using a demo version of the Elecard stuff (StreamEye, etc...), which is typically limited to 500 packets, a lot more data will be processed because elementary stream "packets" encompass a lot more data than MTS ones.

    Chris
  • Announcing version 2.3 of StreamParser. This is a fairly significant update. There are numerous refinements and improvements. The major new feature is a new analysis tool based on the JM decoder. It can provide lots of information about a stream; like frame quality metrics, quantization tables in use, deblocking filtering, and many other things. It replaces h264_parse in the previous version (which had too many limitations).

    Many thanks to Lee Powell and Nick Driftwood - they were super helpful in testing all of the new features.

    As always, remember to uninstall previous versions first.

    Chris

    StreamParserInstall.zip
    2M
  • Version 2.4 of StreamParser is now available. @duartix had requested that additional information about video frames be added. This update adds separate statistics about I, P, and B frames to the Information Panel. This should be useful for determining how bandwidth is allocated between different frame types. Thanks for the suggestion @duartix!

    StreamParserInstall.zip
    2M
  • Brilliant idea.

  • You are too kind.

    Thanks! This shall be put to very good use!

  • Very nice idea! Thanks.

  • What am i missing , ive been sittin here for 20 min and it wont install or open up or what ever.. can someone help.

  • We'll need more information - saying nothing works isn't much to go on.

  • @jclmedia it worked for me.

  • You have to unzip the file into a folder and run setup. Also, it's a Windows program - I don't think it will work on Apple computers. Also, if you don't have .NET installed it will need to download it from Microsoft - so you will need an Internet connection.

  • Announcing StreamParser 2.5.

    This version has many new features to help analyze the quality of video streams.

    Also attached is documentation. Yep... you read right - there's actually some documentation now!

    Chris

    StreamParserInstall.zip
    2M
    StreamParser.pdf
    869K
  • @cbrandin Wow!!! It's very useful! Thanks for your hard work. :-)

  • Indeed, thanks Chris, for providing such an indispensible tool for AVCHD patch testing. The elemental stream macroblock statistics log in the latest version comes just in time to fill in the gap, not that Elecard's Streameye Slim is no longer available.

  • I've put up a wiki page for Streamparser here: http://www.personal-view.com/faqs/streamparser

    The attachments below are only there for the wiki page - nothing to look at.

    Pic4.jpg
    658 x 579 - 100K
    Pic1.jpg
    672 x 403 - 88K
    Pic2.jpg
    672 x 422 - 82K
    Pic3.jpg
    672 x 354 - 88K
    Pic1.jpg
    672 x 416 - 96K
    Pic2.jpg
    672 x 448 - 96K
    Pic3.jpg
    672 x 354 - 84K
    Pic4.jpg
    678 x 626 - 82K
    Pic5.JPG
    746 x 398 - 60K
    Pic6.JPG
    746 x 272 - 41K
    Pic5.JPG
    746 x 398 - 72K
  • @cbrandin Chris, This is a wonderful program now. Just wanna say a big thanks to all the hard work you put into this.

  • StreamParser 2.6 is now available.

    Per VK's suggestion I have added a "Snapshot Report" feature under "File>Generate Snapshot Report". This creates a graphics short-form report suitable for posting on forums, etc... The report generated when StreamParser in in Frames mode looks like this:

    image

    A similar report for Time Mode can also be produced.

    Other changes include a user guide that can be accessed by clicking on "Help>User Guide".

    Other changes include minor bug fixes and prettying-up some windows.

    Chris

    Pic5.JPG
    746 x 398 - 60K
    StreamParserInstall.zip
    3M
  • Added a thumbnail to the Snapshot report when the Stream Player is Opened and not detached:

    image

    Chris

    Pic5.JPG
    746 x 398 - 72K
    StreamParserInstall.zip
    2M
  • This just keeps getting better and better. All serious testers should be using this.

  • @cbrandin

    Thanks for your incredible hard work, Chris! :-)

  • Can we donate for your hard work as well? Your program is fantastic.

  • Thanks, Chris!

  • @mpgxsvcd

    You certainly can donate to VK!

    Thanks for the kind words.

    Chris

  • Thanks Chris. Adding my voice to the choir, your tool is invaluable for settings development.

    Today I think I might have spotted a bug. On a 30p HBR video it looks like a STEP click on the viewer isn't in synch with the frame cursor on the timeline. Instead it looks like the cursor is moving at half speed. The shutter speed was at 1/2.5s so I set it for GOP 12. However you can see it's got new content at frame 6. Here: http://www.mediafire.com/?q8noc2b5a6zcag9

  • @duartix - Thanks for making the file available.

    I looked at the file with the JM-SP decoder in StreamParser and got this:

    image

    The problem is that the clip has many invalid elements in it and it is confusing the stream player. I don't think I can do anything about it. The problems are:

    1) Fallback matrices are being used - resulting in very low frame quality.

    2) There are empty (I assume - the headers are invalid so it's hard to tell) frames that contain invalid parameters per the H.264 spec.

    3) The only frames (almost) containing data come in pairs - I frames and the P frames immediately preceding them. It seems that there are a couple of extra lone P frames in there as well.

    4) There are huge numbers of skipped (not encoded) macroblocks - even in frames that are actually encoded.

    You can look at this yourself by first creating an elementary stream file ("Tools>Create Video Elementary Stream File"), and then running the JM-SP decoder ("Tools>Decode Elementary Stream File (JM-SP Decoder)").

    Chris

    BadFrames.jpg
    658 x 588 - 87K
  • :)

    Thanks Chris, that would be the best news of the week!!!

    You're basically telling me I'm doing everything as I should! LOL.

    It's really not a bug, it's a feature! ;)

    It's the way my settings were supposed to work, although the IQ has it's glitches so it's not totally there.

    If I find that problem with a well behaved file, I'll post it here too. Many many thanks!!!

    BTW, is QST = Quantizer Scaling Tables? And what is DC?

  • DC is the effective QP value within a macroblock for the DC coefficient after scaling tables have been factored in. The values that show up are impossible - so there is something terribly wrong with the clip.

  • Can the fact that the Scaling Tables for P-Frames were a bag full of 0xFFFF justify it?

    I was trying to make them as small as possible!

  • Yeah, that's why it's happening. Also, the I frames are a little strange - the even lines are encoded with normal matrices and the odd lines are encoded with the 255 matrices. Interlaced I frames are actually two slices - one I slice and one P slice. If you run the JM-SP decoder with trace on the trace file shows these slices separately and you can see the matrices used.

  • I believe that was an early test where I was trashing both P and B with Scaling Tables while keeping I at top quality. It's interesting to see that in that case, the encoder uses the scaling tables for I instead of P, even though it is a P-Frame.

    However there must be some kind of extra-reference to earlier P-Frames in that P slice, because this caused a lot of shimmering.

  • @cbrandin I'm currently use AVSPmod macros to generate user friendly avisynth scripts and would like to automatically detect whether the MTS-file is interlaced or not. How did you do this with streamparser. AVSPmod macros are based on python programming language and there might be the possibilitiy to read out a special location within the MTS-file.

  • @cbrandin: I was wondering if you could add just one more column to your I/P/B size summary saying how much percentage of the total bitrate was spent on each of these types of frames considering the GOP structure. Thanks.

  • @cbrandin - "Interlaced I frames are actually two slices - one I slice and one P slice."

    Do you know whether the P-slice in an interlaced I-frame uses the Inter Scaling Table for P-frames, or the I-frame Inter Scaling Table?

  • I don't know - they are the same in the GH2. You'll have to load custom tables that are different to figure it out.

  • Cheers Chris, it is me, you worst unsatisfied client ever... ;)

    Is there a possibility (in the Elementary Stream Decoder) to report Average Quantizer? It would be a boon for VBR developers and would almost instantly kill my sporadic needs to use StreamEye...

    TIA.

  • Hey Chris, absolutely stunning tool thanks.

    One thing, can you avoid writing the .YUV file when decoding via JM, or make it optional (disabled by default)? This contains uncompressed frames and takes a lot of disk space & time to write, especially for longer frame counts. It's not actually used in the analysis right?

  • Cheers Chris,

    This is from one of my Timebuster 24p clips. Is it possible that the player in StreamParser is one frame ahead of the cursor? It's showing heavily posterised I-Frames when these should be the ones with good quality, and the frame sizes supports it.

    http://www.mediafire.com/?y235j1nvxbp5sfd