Main

July 31, 2009

Dog Days of Summer bring in some new features

We've just launched a couple new features on Media Mill. The first and most exciting is the ability to do video recording directly within your browser. Just visit the "Record Video" page to give it a try. Even though it's on our main site, we're still considering this feature a bit "beta" - let us know how it works for you.

We've also revamped the way UThink blogging integration works, to support the new version of Movable Type that UThink is using. Check this help page for more information.

Finally, we've upped the limit on web-based uploads to 75megabytes. We're continuing to monitor the load that this puts on the server, and will hopefully be able to increase it a bit more.

January 28, 2009

MOD, VOB and TOD oh my!

I've got beta support for MOD, VOB, TOD and MPG files up and running, and I'm looking for some folks to test. If you're interested in giving it a try, please let me know.

November 26, 2008

Minor Bug fix

I just made a minor fix in the Quicktime caption convertor. There was an issue that could result in missing spaces at points where the caption had line breaks. If you need to fix that on files you've already created, you'll have to delete and recreate the derivative.

November 3, 2008

Upcoming downtime

Media Mill will have some downtime early Tuesday morning, while we add additional storage. The downtime should start around 8am. The total outage period should be approximately 15 minutes, and hopefully the bulk of media mill (the web interface, serving videos, etc) will be offline for less than that.

Just take it as a sign that you should be out voting! Let me know if you have any questions (mcfa0086@umn.edu).

October 28, 2008

New Features

Here's a short video that runs through some of the features that are about to appear on Media Mill.




What all have we got?

  • Upload from your browser (for small files)
  • AJAXyness - No more page refreshes for deleting archives/derivatives, locking, adding to projects, adding to podcasts, or checking status.
  • Support for setting the author field of podcasts (uses the credits field for individual videos, globally defined in the 'my podcasts' edit page)
  • Revised and cleaned help system - New articles on derivative types, quality levels, quicktime captioning and more!
  • Quicktime captioning support (actually shipped a few weeks ago, but it's official now)
  • Podcast viewing with Mango
  • Performance improvements for page loads
  • Ability to restrict access to only people with X500 accounts

October 7, 2008

Captioning in Quicktime with Media Mill

The Media Mill beta site has a new feature, which allows you to caption Quicktime (and video for ipod/iphone) files, in addition to the existing flash capability.

All you need to do is add your captions to the transcript field (in the same DFXP format as always) and then create a quicktime derivative. The captions will be baked into the file, as a true captioning track. Then you can choose to turn it on and off during playback. In Quicktime Player, it's under the view menu, in iTunes it's under Controls->Audio Options, and on the iPhone it's in the iPod settings menu.

When you're watching it in the browser, just right click (or control click on a mac) on the video and choose "show captions." Here's an example.

August 15, 2008

New Features

Later today, there will be a new Media Mill release that adds a ton of new features. This has been in beta for ages, as we waited for a few pieces to come together. Here's a partial list of what to expect - the links go to videos demonstrating the features:

Plus lots of minor tweaks and bug fixes.

So, what's still on the todo list?

  • H.264 flash integration
  • H.264 streaming
  • Revised Help interface
  • Demux mpeg-2 (maybe, possibly, perhaps...)
  • Migrate flash player to actionscript3
  • Modules for submission to third parties
  • More flash options
  • Autodetect interlaced files
  • And whatever else comes up...

August 13, 2008

Retention Policy

Recently we sent out our retention policy, which basically says that after you leave the University, you get 60 days of complete access, after which we prevent you from uploading new content and remove your source files (without touching your derivatives). Obviously we'd love to keep everything forever, but it's just not possible. The bulk of the content on Media Mill is stuff that doesn't necessarily gain additional value by having the originals stored, and the costs are substantial.

Anyways, there was some confusion from the email - we have no intention of actually billing folks for the storage space they use. We only mentioned it because we wanted you to know how generous CLA-OIT is.

August 10, 2008

Downtime report

Media Mill went offline around 6:30am on Sunday, August 10th. Service was restored around 1:45pm. This was due to another power failure at the WBOB data center. For more information on University service outages, you can always visit the OIT system status page.

July 30, 2008

FLV as a source file

I've just added support for FLV files as source videos, rather than just as a destination. Although it's available to anyone at the moment, consider it beta. I'm relatively concerned that this is going to break other formats, in which case I'll need to disable it. We'll see though...

July 24, 2008

Storage upgrades complete

You may have noticed some notices about downtime this week and last, due to storage upgrades. I'm happy to say that the upgrades are now complete, and we're in good shape to continue growing.

For the technically minded, the first round of downtime, last week, was necessary in order to physically upgrade the RAID array. The second round was necessary in order to actually add the storage to the Media Mill SAN pool. In the future, upgrades should be able to take place in less time, as many of the tasks performed this time were one-time only.

July 2, 2008

Downtime

As some of you surely noticed, MM was down for nearly 24 hours from noon on July 1st until 10:00am on July 2nd.

This was due to the fire and power outage at the WBOB data center. Sorry for the inconvenience.

August 20, 2007

One click to Uthink

Media Mill Video

Nice..


June 28, 2007

More Milestones

It took us 6 months to reach 15,000 videos served. It took one more to reach 20,000. Cool.

May 21, 2007

Milestones

I'm out of town for a bit, but I wanted to post that Media Mill created its 3000th derivative this weekend. We've actually compressed about 9000 derivatives total, counting the DCL, but this is 3000 Media Mill User videos.

We also passed 15,000 videos served a few weeks ago. That doesn't count owners watching their own videos, and we only started counting in November of 2006. So, that's pretty cool I think.

May 11, 2007

De-Beta'd

Hey look, it's not beta anymore.

Rad.

To get an idea of what all has been rolled into this new build, check out my post from last week. Everything except GTV is now live. Cool.

As always, let me know if things are busted up.

May 5, 2007

What's in the pipe

I was just out walking and was contemplating all the new features in the next release of Media Mill. I figured I'd give a roundup of what's coming (hopefully in the next two weeks).

  • Support for audio-only files (transcoding to AAC or FLV)
  • Thumbnail browsing interface (browse by thumbs instead of clip titles)
  • Captioning support for Flash files
  • GopherTV support (may not be enabled in the release)
  • Automatic widescreen detection (including anamorphic DV files) ... hopefully
  • Various bugfixes

Hopefully this will be rolled out towards the end of this week, depending on how implementation of the anamorphic detection goes...

March 13, 2007

Media Mill Overview (again)

I've done another short overview of Media Mill, including some of the new features. Take a look.

March 11, 2007

Downtime Today

In case anyone is wondering, the downtime today was just due to the CLA-wide outage for some power work at the data center. Everything is cool with Media Mill.

March 8, 2007

If only I could....

Just so it's out there - I intend to add a group of presets for encoding "Screencast" videos - optimized for progressive sources with very long distances between I-frames. Unfortunately, because screencasts can have funny resolutions, I want to do the resolutions as percentages of the original, and that feature is broken in the current build of compressor. Look for that feature in the future though!

January 31, 2007

Major feature rollout

Big changes in the world of Media Mill today. I just installed a new copy of the codebase, which represents the last six weeks or so of development work. What you get:

  • "my Projects" - you can now establish projects which contain multiple videos. You can share a project with another user and they'll also get all the videos it contains. Then, they can add videos to the project and they'll show up for you as well, automatically. It's like magic!
  • Derivative locking - you can click the lock icon next to any derivative. It'll prompt you for a "lock message." Once a video is locked, it can't be deleted without first being unlocked. The idea is that if you've embedded a video in a blog, or sent links out to all your friends, you don't want to delete it by accident. The lock message is intended to remind you of why it's locked. Clever eh?
  • Drag and drop rearrange - this is automatically enabled in the project interface, and can be selected in the general "my videos" page. Select the "custom" sort order, and then just click and drag to rearrange your video list. The order is then automatically stored on the server as you move things around, so it'll stick when you return. This works best if you "collapse all" first. This should work on all the modern browsers (Safari, Firefox, IE7).
  • Persistent sort settings - Your sort order choice (a-z, date, custom) is now stored in a client side cookie.
  • Improved performance - If you have your videos collapsed, thumbnails won't be loaded every time you refresh the "my Videos" page. Instead, they'll be loaded dynamically when you click the disclosure triangle. For those of you with lots of derivatives, this should improve performance substantially. Additionally, some javascript has been stripped, some has been consolidated, and the rest has been compressed.

I think that's about it. There are loads of other minor tweaks.

Please let me know if you run up against any issues with these new features. Thanks!

January 22, 2007

Shaking hands with the Devil

Despite what you might think, there is a lot of esoteric thought that goes into writing code. Programmers agonize over the elegant way to do something, and feel guilty when resorting to a "hack" - even if the end result is identical. That's not to say that bad code doesn't get written, but it's often because outside constraints demanded it.

Today I had to shake hands with the devil.

It used to be that no two browsers could be expected to perform at all alike, particularly in regards to Javascript. Netscape and IE would do things entirely different. This meant that pages often had whole decision trees to display different code depending on the browser. This was fundamentally The Wrong Way (tm) to do things, as it meant there were tons more places for code to break, and it left the code ugly and hard to maintain. As time has gone on, browsers have gotten better and more flexible, so oftentimes code will render nearly identically across browsers.

Usually.

There's been an issue for a while with the rendering of the "window blind" effect on Internet Explorer. This isn't a problem in Scriptaculous (the library that provides the effect) but rather something to do with the nested divs on the Media Mill myVideos page. One of the features in beta testing right now allows you to drag and drop your videos to arrange them however you like. Adding support for this has exasperated the rendering issue on Internet Explorer 7, such that blocks of text would disappear and reappear as you moused around the screen.

I spend the better part of this afternoon trying to solve the issue, but my CSS-kung foo is insufficient, and so I've resorted to something I had long tried to avoid. I added an IE-specific hack. IE users will no longer get the pretty window-blind effect. I hope that this is a small price to pay for the added coolness of dynamically rearrangeable videos, but it still makes me sad. Cry.

January 16, 2007

New Xserve in the cluster

The new Xserve, mentioned in the last post, is now online. Let me know if any odd problems crop up. Otherwise, just enjoy the massive performance boost, particularly with Quicktime encoding.

January 15, 2007

Holy Speedy Compression Batman

After a bit of a technical delay, our Intel Xserve is up and running and boy is it fast. I'm still doing shakedown testing on it, but for fun I threw it in the production cluster for a bit. For Quicktime jobs, you'll see approximately a 250% speedup in compression jobs, as we've now got six processors churning on those jobs. Flash and Windows Media jobs can't be split between multiple machines, but it means that we can now process six Flash jobs simultaneously, rather than two. That's pretty significant.

As I said, the server is still going through some testing, so I'm not going to leave it in the cluster tonight. I need to reengineer all of the Flash presets in order to work properly, among other things. Hopefully we'll have it up and running for all of you later this week.

January 6, 2007

Media Mill Downtime

Hey all - Media Mill was offline for a few hours earlier today due to a hardware issue. We're still investigating the cause so that we can be sure it doesn't happen again in the future. On a side note, we're very close to bringing our new compression server online, which should significantly speed up compression jobs.

December 28, 2006

Slightly updated uploader

I just updated the uploader slightly with some new dialog boxes. Nothing should have broken, but please let me know if it has...

December 12, 2006

Sticky boxes

More new features, hizzah!

Your MyVideos page will now remember whether you left the boxes expanded or collapsed. New videos will default to expanded. You can also use the "expand all / collapse all" links to reset everything.

It's ajax to the max, I'm so web 2.0 ...

December 5, 2006

Thumbs up to sharing

So, sharing seems to be working pretty well. I'm going to take Media Mill offline very briefly tonight in order to install the sharing feature. By tomorrow morning, everyone should see a little icon of a person, something like this:

Addshare

Just click the icon and then enter the name or x500 of the person you want to share with. Easy peasy!

Update: Sharing is live now

December 2, 2006

Sharing feature ready for testing, beta testers needed

Hey All - I've got the sharing feature more or less finished. This will allow you to share a video from your "MyVideos" page with any other media mill user. They'll then be able to make derivatives or do anything else, just like if they uploaded it themselves.

I don't want to roll this out without doing a little beta testing first. Is there anyone out there who'd be interested in giving it a shot?

Image 056
Whiteboards help us play ...

November 28, 2006

Tooltips on Derivative page

I just rolled out a minor change to the "request derivative" page which shows tooltips for the Quicktime Streaming types. I'm hoping this will make it more clear to the folks who are creating the QTSS types without having access to a QTSS.

Now that I have tooltips more or less working, I may add them to another parts of the site. Do folks like these? I haven't decided how I feel about them yet.

Sample Code bug fix

I just fixed an issue which prevented our "sample html" code from displaying properly in Windows Notepad. It should come up properly on all platforms now.

November 15, 2006

Podcasting is Live!

Everyone should now have a "My Podcasts" link on the left side of the screen. I've also posted a help document explaining how to use the podcasting feature. I'll try and get more information up shortly. For the time being, just let me know how it goes!

Expand/Collapse Video Blocks

If you have more than 10 videos in your My Videos page, you'll now notice that the video blocks are collapsed by default. Click the little triangle to expand them. Gasp at the amazing web2.0y effects!

Cool? Stupid? Let me know.

November 14, 2006

Flash Crash Dash Bash

There's an issue with the FLVPlayback flash component crashing Windows browsers when displaying videos greater than around 200megabytes in size. I've just pushed a workaround that will use a different player for those very large flash files. It's not quick as full featured a player as our standard component, but it has the advantage of not locking your computer.

Quicktime Embedding

Public Quicktime movie links will now also show a "sample html" link. So, if you're of the Quicktime persuasion, you can now embed your content. There are still some issues embedding in Internet Explorer, due to the ActiveX lawsuit issues. The workaround involves document.writein's in javascript, and I'm not sure I want to go there.

November 2, 2006

Media Mill Queue

You may have noticed that jobs have been spending a lot more time "waiting," lately, instead of processing right away.

We've seen a pretty big spike in usage, and so the cluster has been churning 24 hours a day for the last week or so.

The good news is that we've got another server on order, which will triple our processing capacity. The bad news is that it's still a few weeks away. So, at this point, you're stuck waiting I'm afraid.

Oh no, abandoned blog!

Ok, so, my bad. I totally forgot about this blog. I was standing in line for coffee thinking "I wonder how I can let people know what's up with Media Mill ..." and then it struck me! I have a blog for that!

Onwards.

October 4, 2006

Universal Uploader

The Mac OSX Uploader is now a Universal Binary, so those of you on Intel Macs should download a new copy. It's not actually any faster, but I feel better about it now...

175Px-Universal Binary Logo

October 3, 2006

Email Notification

This feature actually appeared a few days ago, but I forgot to blog it. When you submit a derivative, you can now check a box asking for email notification upon completion. The email will go to your UMN x500 address. Pretty nifty, eh?

We'll hopefully be gearing back up for some more significant Media Mill features over the next few weeks. Sorry things have been a bit slow - we've all gotten caught up in other projects. Check Discrete Cosine for more info...

August 8, 2006

Media Mill Demo Video

I've put together a short video to walk you through adding a video and making a derivative. My first try at voiceover work...









July 27, 2006

HD Preset

Wow, it's all zip zap zoom with the updates today. Thanks Dan!

We've now got an HD Quicktime 7 preset. This will only show up if your source media is at least 1440x1080. You'll get a 1920x1080 H.264 file with a rather chunky bitrate - perhaps as high as 8mbits/sec. For reference, this is on par with the HD videos you can get from Apple - they tend to range from 8-10mbit, and so became my target when creating the preset.

Please submit comments if you think this preset should be tweaked in any way. I don't have a ton of HD footage to test with, so it's my best shot at balancing Ultra-Quality with reasonable compression.

Windows Media Presets

Our derivative support has grown again - you can now create Windows Media 9 video derivatives, in the standard Small, Medium, Large and Very Large formats.

These are all WMV9, single pass. With the exception of the Very Large format, they are all constant bitrate. This may switch down the line, when I have time to tweak the variable bitrate (VBR) presets for each quality level.

The WMV9 is more or less on par with the Quicktime7, and definitely a step above Flash. Only in the most compression-challenging scenes does Quicktime 7 (h264) take an edge over Windows Media (VC1).

July 26, 2006

Performance Enhancements

Some good news about performance for once.

As I mentioned earlier, I had tracked down the Flash performance hit to either Compressor or FlixExporter. I now think it's a combination of both.

Essentially, if we tell the Compressor software to store its temporary files outside of the default location, disk accesses for the Flash plugin go through the roof. While this fundamentally seems to be a bug in Compressor (and I've reported it to Apple), the FlixExporter seems to blow it way out of proportions. Flash jobs were using, at most, 40% of the available CPU time.

For now, I've switched to a secondary Flash encoder - this time, the default Quicktime plugin which comes with Flash 8 Professional, instead of the separate On2 product. You get less control over the final video this way, but I think it'll be Good Enough(tm) for now.

While this plugin still stuffers from the Compressor issue, it's at most suffering from a 12% performance hit, instead of a 60% (or more) performance hit.

As an aside, the reason that we're using a non-default storage location for the compression cluster is that it allows us to get rid of the annoying "preparing video" step. During that step, your whole video had to be copied off of the RAID array and onto a local harddisk. This was a major time hit, and also meant we had to set aside extra storage on the local machine. By placing the storage on the RAID, your video starts processing as soon as you hit submit - on larger jobs, this could save you up to 10 minutes of processing time.

New Feature: Download your Original

When you access your "My Videos" page, you'll now see that you have a download icon attached to each original video. This allows you to download the original video file which you uploaded to Media Mill, totally unmolested.

This file is only available to you - the link requires that you be authenticated with your University username and password.

July 25, 2006

Flash is Faster

So I nailed our Flash issue down to a bug in either Compressor or FlixExporter. The good news is, I've been able to work around the issue. I'm going to continue working to track the issue down, but until then, Flash encoding should be between 2 and 4 times faster. Hoorah!

Why is flash so slow?

Short answer: I have no idea. Read on.

Continue reading "Why is flash so slow?" »

July 24, 2006

Embedded Flash!

Follow the link for my test of YouTubeyness!

Continue reading "Embedded Flash!" »

July 21, 2006

Regarding Job Completion Information

If you've played with Media Mill much, you may have noticed that the percentage-complete indicator for your job doesn't update that frequently. The reason for that is that we only cache that information once every 60 seconds (give or take). So, the percentage you see could be up to a minute out of date. This is because it actually takes quite a long time to get status information out of the compression cluster. When we initially deployed Media Mill, all that information was queried on every page load, and it took *forever*. 60 seconds seems like a reasonable tradeoff. Doing it more frequently would just deprive the actual compression process of cycles, making your job take that much longer.

Make sense?

Preset Revamp

I've just finished revamping all of the Quicktime 7 download presets. While working on testing for the upcoming WMV presets, I couldn't help but notice that the WMV9 looked a whole lot better than the Quicktime7 (h264) that we were generating. Well, perhaps not a whole lot better, but better nonetheless.

After poking around a bit, I came to a few conclusions. First, a lot of the perceived quality advantage of WMV9 is just because they're oversharpening the decoded results on playback. There isn't actually more detail, the edges are just more vibrant. Next, I realized that WMV9 is not using a fixed keyframe cadence (at least, as best as I can tell). I had previously had our QT7 presets set to keyframe every 30 frames. I've now switched that to automatic.

Finally, I don't think that WMV9 actually maintains a fixed bitrate, even with the CBR setting. I think it's more like semi-fixed - it won't exceed the cap, but may shift bits around a bit. In order to replicate that effect, I've switched the download presets from using a hardcoded bitrate to using a spatial quality setting. I had previously been wary of this, as it can result in files that are of an unexpected size, but in my testing, I think the new presets will deliver file sizes about on par with the old settings, while providing higher quality results.

Why you may ask, does it provide such better results? By using a spatial quality setting (temporal stays at 50%), the compressor is able to shift bits around, giving a few more bits to a complex scene and giving less bits to a less complex scene. This means the video is much less likely to "fall apart" and start showing big macroblocks.

The streaming settings remain unchanged, as I still believe that a constant bitrate is a high priority for good streaming.

Please, let me know if you think the new presets are good, bad, or still lacking.

Windows Media 9 support

We've added support for Windows Media as an import format. You can now submit all of those WMV files you have choking your harddisk, and convert them into Quicktime or Flash.

I'm also in the last stages of testing WMV9 presets, so that you'll be able to generate Windows Media video. I must say, I'm very happy with the quality of the Flip4Mac export component. It gives me plenty of control and is far more userfriendly than the On2 flash encoder we're using.

Why aren't we doing H264 for the iPod?

Anyone who's used the iPod preset has probably noticed that we're using the older MPEG-4 codec, instead of H264.

Initially, this was done because up until Compressor 2.1, the H.264 that was generated wouldn't work on the iPod. Compressor 2.1 added an iPod preset, but unfortunately it comes with a little bug. Apple is investigating the issue, and when the issue is resolved, I will switch the preset to H.264. Until then, we have to sacrifice a little video quality in order to preserve the audio.

Welcome!

This blog is intended to provide updates about the MediaMill project (http://mediamill.cla.umn.edu). I'll try to post here when changes are made to the site, or when I want to discuss future directions.

Please post comments with your feedback!

-Colin