Will’s Miro dev blog

Here’s the list of channels I’m currently subscribed to with Miro:

• Timo’s HD movie trailers - feed | subscribe
• What you ought to know - feed | subscribe
• Webbalert - feed | subscribe
• XPlay - feed | subscribe
• Ubuntu Developers - feed | subscribe
• Fedora TV - feed | subscribe
• CSS-Tricks - feed | subscribe
• GitCasts - feed | subscribe
• Google Tech Talks - feed | subscribe

Subscribe to them all!

Progress went well this week. We’ve been working on the missing functionality, fixing bugs, removing dead code, cleaning up existing code, and generally progressing on Miro 2.0 development.

Last week, I wrote up the list of must-have features before Miro is ready for testing. We finished the following items:

• diagnostics dialog.
• open_file needs to be implemented in the OSX Application class for “show” and other things to work.
• fullscreen playback
• hook up volume control to playback renderer
• Make playback take over the whole window on GTK
• search tab
• show/hide details - needs to be fixed on OSX
• download tab: pause all, resume all
• download tab: Top bar display, download / upload rate

And fixed a bunch of bugs. Ben did some optimization work on OSX, too. Some of those are new items added to the list in the last week. Some of them are smaller chunks of existing items.

We have the following list of things to do still:

• Preference window
• Chrome Search
• within channel search and save
• ff / rw / seeking
• download tab: External Downloads section in main view
• show/hide details - Needs seeders and leechers info for torrents
• Channel Settings
• overlay playback controls on OSX (including fullscreen button)

You can follow the progress on the timeline.

Things are progressing. I landed the last of the menu handling changes, Ben did a bunch of Mac OSX tweaks and Windows tweaks, there’s been work on playing videos, Ben re-worked how we deal with resizing thumbnails so it no longer uses imagemagick and no longer generates lots of files in the icon-cache, and there have been a lot of bug fixes as well.

On Tuesday, a bunch of us got together in Worcester for the Miro Summit and worked out some things including the plan going forward. From that discussion, Nick wrote up a list of must-have features that need to be re-implemented before we want to get functional testing going. This list is likely to change, but here it is generally speaking:

• Preference window
• Chrome Search
• search tab
• within channel search and save
• search all for search engines
• fullscreen playback
• ff / rw / seeking
• hook up volume control to playback renderer
• download tab: Pause all, resume all, cancel all — Top bar display, download / upload rate — External Downloads section in main view
• channel details view

From that list, Chris Webber is working on a patch for channel details view which I think is close to done. I think I’m going to tackle Preferences next. I’m not sure what other items are spoken for. I’m also not sure how much time it will take to tackle all of these–maybe a month?

That brings me to my next two points:

First, I know it’s taking us a long time to get things done, but it’d be faster if we/I didn’t have to answer issues along the lines of “are you done yet?” and “i tested xyz nightly and the following things don’t work…”.

I really appreciate the fact that you all are eager to help out with Miro development and I really apologize for the frustration that comes with not being able to do anything and having to wait around for a while. We’re working as hard as we can, but it’s just going to take time. I’m in this weird position where I want to maximize the time I’m spending doing the work and minimize the time I’m spending talking about it and managing it, but I don’t want to leave you in the dark about where we are. It’s a tough thing to balance especially where I’m trying to minimize the time I spend thinking about balancing.

Oddly, I bumped into this post on 43 Folders which I think is pretty relevant here.

The second thing is that we’ve had a lot of help from Chris Webber over the last few months. He’s been working in his spare time to come up to speed with the new widget code and help re-implement functionality. His efforts are fantastic, the quality of his work is solid, and he’s absolutely making a big dent in our ability to finish the work faster. You can see the efforts of his work here–I refer to him in the checkin comments as “Chris”, “Christopher” and “Chris Webber”. Thank you Chris!

That’s it for this update!

I apologize for skipping last week’s status of trunk.

Trunk is still unstable. I consider “trunk is stable” to mean that most things are re-implemented and it’s usable. We’re not quite there yet. I’m seeing occasional crashers (most of them my fault so far), there are a bunch of features that haven’t been re-implemented (site search, channel search, playing through a playlist, setting preferences, …), and there’s still ui nits that need ironing out.

We’re moving along, though it seems as if the progress comes in waves during the week. I think part of this is that we’re all using git over svn and batching commits. I know I am.

I was gone all last week at OSCON, but I’m back again to help out with the push to Miro 2.0.

You can follow along in your story book with the Trac timeline.

This is awesome news! Hooray for Mozilla!

I eagerly look forward to the many, many things Mozilla is enabling us to do.

Play OGG!

Edit: Another blog entry espousing the goodness of OGG support in Firefox!

It was a really great day today. Talked with Songbird and Thunderbird people about our projects in relation to contributors and outside contributions.

There’s a lot of interest in a plugin system for Miro–it’s definitely on my list of things to work on in the near future. There’s also a lot of interest in a user interface that’s tv-friendly. That’s something I think about a lot, but it’s not something in my immediate queue.

I talked to Dave Camp on the work he’s doing on the new embedding API for Gecko–it sounds pretty cool.

Met a lot of happy Miro users. There are a lot of people out there rooting us on which is very exciting.

That’s the end of OSCON for me. I’m going to spend Friday hanging out with my sister and then I fly back on Saturday. It’s a pretty incredible conference all things considered. Met a lot of great people doing a lot of really fantastic work. It’s very inspiring.

I spent the day in the Mozilla booth in the Expo hall. I talked to 30 or 40 people, I think–after lunch it was kind of a blur; after dinner it was definitely a blur since the expo hall was serving free beer (or “free beer as in beer” as I said a couple of times which I thought was pretty funny, but went over like a lead balloon). There are a lot of Miro users at OSCON, a handful of people who have never heard of it, one or two people that thought it was still named Democracy Player, and I met Michael Frank who contributed code to Miro at some point before I was with the project!

I also talked with a lot of Mozilla people who came and went over the course of the day. Listening to Ben and Taras talk about static code analysis was really exciting. I talked to Stephen, Peter and Rob from Songbird. We spent some time talking about where Songbird and Miro overlap and what kinds of things we can work together on. The immediate result of this is that I’m going to start hanging out on their IRC channel and probably start looking over their code base. Additionally, Peter is working on a system of identifying media from RSS/Atom data. I’m hoping to help out since this is a problem Miro has, too, for some feeds.

I also go to see Asheesh and Nathan from Creative Commons and we had time to catch up on life and projects.

I went to the BoF session on addons for XUL-based applications. We talked about pain points in the extension-development process. Overall it was really educational both from the standpoint of how to write XUL-based application extensions as well as what kinds of things we need to focus on when we create a Miro plugin system. Songbird has an extension system–we should definitely look at what they’ve done given that they’re very close to us in application space. I also went to the BoF session on the static analysis project Ben and Taras are working on. Less because it’s applicable to anything I’m probably ever going to do, but more because it’s just mind-bogglingly interesting. They call it Dehydra.

That’s it for today. Tomorrow should be interesting, too.

I went to the convention center to check in today and run some errands. I have an Expo pass, so I’m not able to go to the tutorials, sessions, keynotes or some of the other things that were happening Monday and Tuesday. Even so, there seems to be plenty to do.

Amongst other things, I talked to someone about his Eee PC, I did some Miro hacking, I drew a Charlie Brown stripe on my laptop cover (it came out awesome!), I talked to iSneeze on IRC about plugins (I need to write up a post about plugins), and I discovered that there are 5 Starbucks locations on the 8-block walk from the convention center to one of the nearby Kinkos locations.

I was going to help Jay with the Mozilla booth setup, but they finished before I got in contact with them. Instead, I went to the FSF Portland Pizza Party which was really great. I talked with Mako and Asheesh both of whom I’ve known for many years from PyBlosxom. I also met Karen and Aaron from the SFLC and Deborah and Joshua from the FSF. I also met Gerv from Mozilla, too. And John Eckerman and a man named Charlie and a man whose name I think was Craig.

Everyone I talked to fell into one of two groups:

1. Either hadn’t heard of Miro or had, but didn’t use it
2. Avid user of Miro

I didn’t meet anyone who had heard of Miro and didn’t like it. It sounds like people are happy with Miro and look forward to the future of Internet video with open standards without gatekeepers that we envision. Also, several people mentioned that the content on the Channel Guide just keeps getting better.

While at the FSF pizza party, I became an associate member. The work that they’re doing is really beneficial to us all. They’re pushing for open media standards adoption, pushing for fixing the patent system in regards to software, fighting against DRM, and a variety of other causes. It’s important work that needs to be done for video on the Internet to be open to everyone.

I’ll be spending the next two days in the Mozilla booth. If you’re at OSCON, drop by and say hi!

I’m heading to OSCON 2008 and will be hanging out in the Mozilla booth talking about Miro things, Mozilla things, happy things, and whatever other things.

When I’m not hanging in the Mozilla booth, I’m hoping to be doing some Miro hacking (and possibly some PyBlosxom hacking–we shall see).

I’ll be wearing Miro and Miro-related t-shirts until I run out. Then I’ll be wearing other things.

If you’re planning to be at OSCON, stop by the Mozilla booth and talk to me about things you like, things you dislike, and favorite channels. I’m also very interested in helping people learn how to help out.

First off, sorry that this status update is so late–I was side tracked on Wednesday and then the status slipped my mind.

In the last week and a half, there’s been a lot of progress in trunk. While we haven’t re-implemented everything yet, we’re doing pretty well.

You can follow along in your story book with the Trac timeline.

I really appreciate your patience. I think we’ve got another few days to a week before testing builds is useful where “useful” is defined as “you’ll start having a good chance of catching things we don’t already know about”.

On a side note, one year ago (i.e. the 12th of July, 2007), I started working at PCF. It’s been a really exciting year! Whee!

Following Paul’s lead, here are some cloc stats for Miro in trunk:

willg@mercury:~/pcf/miro/trunk/tv$ perl /home/willg/Desktop/cloc.pl .
    3468 text files.
classified 3457 files
    1644 unique files.
    2763 files ignored.

http://cloc.sourceforge.net v 1.04  T=20.0 s (35.2 files/s, 9438.1 lines/s)
-------------------------------------------------------------------------------
Language          files     blank   comment      code    scale   3rd gen. equiv
-------------------------------------------------------------------------------
Python              286      9689     10059     53376 x   4.20 =      224179.20
C/C++ Header        265      7941     14412     31565 x   1.00 =       31565.00
C++                  83      5474      4591     27832 x   1.51 =       42026.32
C                     9      1159       889     13119 x   0.77 =       10101.63
Javascript           17       420       557      2699 x   1.48 =        3994.52
CSS                  11       391       476      2451 x   1.00 =        2451.00
IDL                   7        18         0       486 x   3.80 =        1846.80
XML                  13         2         3       275 x   1.90 =         522.50
Bourne Shell          8       136       319       248 x   3.81 =         944.88
make                  1         7         0        94 x   2.50 =         235.00
HTML                  4         2         2        67 x   1.90 =         127.30
DTD                   1         0         0         3 x   1.90 =           5.70
-------------------------------------------------------------------------------
SUM:                705     25239     31308    132215 x   2.41 =      317999.85
-------------------------------------------------------------------------------

Over the last week I’ve been coming up to speed on the architectural changes that occurred when Ben landed the new widget code. I’ve also been hooking up menu items to their respective behavior and fixing bugs on the Linux and Windows platforms. In many cases, I’ve been re-implementing the behavior using the new messaging system which has required me to read through the “old” code and figure out what the behavior used to be. Progress was slow at the beginning, but is picking up now.

You can see checkins progress in the Trac timeline.

There’s still a lot of work to do to get things working again, but things are progressing.

So, why all the trouble? Why not just leave it as is? Off the top of my head:

1. Miro’s UI is no longer rendered using HTML templates. w00t!
2. It looks like overall memory usage is lower by around 20%.
3. Memory usage of Miro when displaying feeds with lots of items scales much better.
4. Miro’s faster at displaying feeds with lots of items (where “lots” is defined as > 50).
5. Miro on Windows is no longer a XULRunner application; instead we’re embedding XULRunner for web-browsing. XULRunner is a great platform, but this change makes Miro a Python application on OSX, Linux and Windows and we can unify our toolset. That’s a huge win for us and reduces the amount of work it takes to maintain all three platforms.

Regarding the performance gains, I’m seeing those on Windows and Linux, but I definitely haven’t spent a lot of time doing rigorous measurements. Treat them as if they were wild unsubstantiated rumor. I haven’t used Miro on Mac OSX enough to notice anything there, yet.

Getting there…!

Just as a word of warning, if all goes well, Ben will be merging into trunk what he and others have been working on over the last few months in the next day or so. When that happens, trunk will become very broken and will remain that way for possibly a week as folks work on stabilizing the new user interface code and reimplementing a bunch of features… silly things like being able to play videos.

If you’re tracking trunk or using nightlies, you should seriously consider waiting a week to update.

You might ask, “Gah! What a bunch of idiots! Why are they doing this?!” The answer is that the widget overhaul is very badly needed and it’ll fix one of several big performance issues that Miro currently has. It’s a big change and it’ll cause a big mess for a short while, but it’s going to make a big big big difference.

In order to do this quickly, we’ll be focusing pretty hard on getting trunk working again for a while. My apologies if any of us seem like we’re dropping off the face of the earth or ignoring questions, concerns, …

I’m at FUDCON10 right now (Thursday, 6/19/2008). I’m interested in meeting people here and talking about Miro and Fedora TV. If you’re here, come find me. I’m currently hanging out in the main FUDCON room in the corner all by myself.

We’ve had a few people contributing patches for the Miro codebase. I decided it was time to add a review flag like Mozilla has in their Bugzilla instance. This makes it easier for us to keep track of attachments that are waiting on reviews.

As such, I added a “review” flag to our Bugzilla instance for attachments.

I thought I’d talk a little bit about contributing patches.

Guess what? You’re a happy Miro user except for one little thing that really annoys you. You head on up to the Miro Bugzilla bug-tracker to see if this is a bug/feature that someone else has reported already. Wow–turns out it has been reported. Not only that, but there’s been some analysis done and some speculation as to a good attack vector for fixing the problem.

So you roll up your sleeves, add a comment on the bug stating you’re going to work on it, set yourself as “assigned”, dust off your favorite editor, skim through the Miro development wiki for the svn repository information and build instructions, and get to work.

A few hours later you’ve got it working on your machine. You run:

svn diff > bugid.patch

to generate a .patch file containing the changes you’ve made.

You visit the bug in the Miro Bugzilla bug-tracker, find the bug you were working on, and click on “add attachment”. You’ll see the following screen:

Deftly, you upload the patch, click on the “patch” checkbox, select “?” from the Review flag dropdown and type in will.guaraldi@pculture.org in the Requestee box.

Then you press the “submit” button!

Will (that’s me) gets an email stating that there’s an attachment waiting for review. I add it to my queue of things to look into. If it’s not something I know anything about, I’ll find someone else who can look at it. Then someone will add a comment to the bug reviewing the patch and … the rest is iterations on that.

If you’re interested in helping out, we’ve been tagging bugs that we think are good for people new to the codebase as “bitesized”. You can see a list of them here.

I was the Q&A person for the June 3rd chat yesterday and I thought I’d post some follow-up and answer some of the questions we didn’t get to.

Follow-up

First off, I thought the chat went well and it was neat to talk to a bunch of people I’d probably never talked to before. It seems that the predominant theme was “when are features landing?” and “what’s coming up?” It’s a little hard to answer those questions because I’ve been pretty focused on the next release that I haven’t been involved in planning beyond that.

Now to answer some questions that we didn’t have time for.

How long have you worked for PCF? What did you do before Miro?

I’ve been working at PCF since August 2007. This is in many ways my dream job with my only issue being that I wished I either earned a bit more money or had a lower cost of living. Other than that, I’m doing what I love doing, I get to hang out with some really great people, and the stuff we’re working on is really important to me.

I’m going to assume “What did you do before Miro?” means “What did you do before working at PCF?” Prior to PCF, I spent 2 years getting a Masters at Northeastern University CCIS in programming language design/theory and software engineering. Prior to that, I worked in the financial services industry at ByAllAccounts, I worked as a contractor for Tallan at Ingram Micro on their international web-site system, and various other software developer positions before that. I’ve been programming for probably 20 years now in various forms, but this is my first FOSS job and the first job where I’ve worked with XULRunner and GNU/Linux-stack components like Gtk+, GStreamer, Xine, Glade, DBus, Glib, GObject, … It’s been great!

shoestring: Video metadata: any plans to see miro actually writing it to the files/being able to edit it?

We’ve talked about making ui changes to allow for changing “metadata” of content, specifically name, filename, tags, … It hasn’t happened yet, though. I think it’s one of many things waiting for the great widget overhaul.

Miro can export feed information to OPML format, but this doesn’t include metadata about content. I don’t know offhand if there are plans to add that or not. There are plans on building an API to allow programs like MythTV and Elisa and other systems like that access to Miro data. That hasn’t happened yet, either. In this case, I think it just needs someone to work on it.

Evan: Is it possible to install miro without bittorrent? I know this question is weird, but in some (many?) companies bittorrent is banned … yet the company is ok with limited internet video usage.

We don’t have builds that don’t have bittorrent in them. It would take some work to decouple Miro from libtorrent and/or disable it and then it sounds like we’d have to provide a separate set of libtorrent-less builds. I don’t think that’s a bad idea, but I don’t think it’s going to happen without a champion who can do the work.

will: (seed question) What do you do when you’re not working on Miro-related things?

A little silly answering my own seed questions, but … so it goes.

Lately all I’ve been doing is Miro-related things. We’re pushing really hard on the next release. We’re really excited about it and we think it’s another big milestone in Miro’s life.

This year, I’m a backup admin for GSoC for the PSF thought I haven’t actually had to do much (yet).

I’ve been trying to finish up work on version 2.0 of PyBlosxom for the last 6 months but haven’t found time and energy to get there. I’ve been able to make some progress, but it seems to be on a permanent back-seat.

I’d really like to help Mozilla on their embedding efforts. I’d also really like to get more involved in gstreamer, Python 3000 and a bunch of other projects.

Epilogue

I think that’s about it. Given that the chat went pretty well all things considered, there will probably be another one in the future and probably more after that.

I live in Somerville, MA, USA and I’d like to organize a Miro hackfest in or near Boston. Possible topics for that hackfest include:

• cleaning up and improving the gtkx11 platform interface, gstreamer/xine use, …
• working on bitesized bugs and working on unittests
• hacking together an interface for Elisa or MythTV
• testing out the fledgling Mozilla embedded API with the gtkx11 interface
• sorting out packaging issues
• other things?

I was thinking we’d do the hackfest sometime in June. Possibly as part of FUDCon10 or in the vicinity.

If you’re interested and/or have ideas, find me on IRC, email me, comment below, or send me telepathic messages of hope.

• (r6949, bug 9908) gstreamer renderer uses gconfvideosink now–much better video output
• (lots of revisions) new user interface
• (lots of revisions) Windows ui looks and acts like Mac OSX ui
• (bunch of revisions) all unit tests pass

Am I missing anything? Let me know in the comments.

Back in December and January, I worked on some patches for Firefox 3 that enhanced the feed preview page. I wrote a post about it back then… but I’m updating that post with recent screenshots and a better description of the work. The previous post was mostly about how great FOSS is.

The patches fell into two big features. First, I added enclosure detection to the FeedProcessor and then modified FeedWriter to show enclosures alongside the entries. This has two huge benefits: it allows you to easily tell if the feed has enclosures and it allows you to see what they are, how big, what type of media, …

Second, I modified Firefox so that it allows you to associate video podcasts with an application, audio podcasts with another application, and all other kinds of feeds with a third application. The benefit here is that you can send media podcasts to an application that handles that well (*cough*Miro*cough*) and regular news feeds to a different application that handles that well.

Screenshot of Firefox 2 feed preview page:

Screenshot of Firefox 3 feed preview page:

Of the two features, I hear the most comments about the first one mostly along the lines of, “I’m so glad I don’t have to view source to see the enclosures anymore!” The second feature isn’t as immediately exciting. The implementation of distinguishing feeds is intentionally simple and there are a lot of corner cases where it doesn’t work very well. Also, there aren’t many applications that can really take advantage of it. I expect this second feature to flourish as Firefox development continues and video/audio podcasting evolves.