Will’s Miro dev blog

Progress went well this week. We’ve been working on the missing functionality, fixing bugs, removing dead code, cleaning up existing code, and generally progressing on Miro 2.0 development.

Last week, I wrote up the list of must-have features before Miro is ready for testing. We finished the following items:

• diagnostics dialog.
• open_file needs to be implemented in the OSX Application class for “show” and other things to work.
• fullscreen playback
• hook up volume control to playback renderer
• Make playback take over the whole window on GTK
• search tab
• show/hide details - needs to be fixed on OSX
• download tab: pause all, resume all
• download tab: Top bar display, download / upload rate

And fixed a bunch of bugs. Ben did some optimization work on OSX, too. Some of those are new items added to the list in the last week. Some of them are smaller chunks of existing items.

We have the following list of things to do still:

• Preference window
• Chrome Search
• within channel search and save
• ff / rw / seeking
• download tab: External Downloads section in main view
• show/hide details - Needs seeders and leechers info for torrents
• Channel Settings
• overlay playback controls on OSX (including fullscreen button)

You can follow the progress on the timeline.

Things are progressing. I landed the last of the menu handling changes, Ben did a bunch of Mac OSX tweaks and Windows tweaks, there’s been work on playing videos, Ben re-worked how we deal with resizing thumbnails so it no longer uses imagemagick and no longer generates lots of files in the icon-cache, and there have been a lot of bug fixes as well.

On Tuesday, a bunch of us got together in Worcester for the Miro Summit and worked out some things including the plan going forward. From that discussion, Nick wrote up a list of must-have features that need to be re-implemented before we want to get functional testing going. This list is likely to change, but here it is generally speaking:

• Preference window
• Chrome Search
• search tab
• within channel search and save
• search all for search engines
• fullscreen playback
• ff / rw / seeking
• hook up volume control to playback renderer
• download tab: Pause all, resume all, cancel all — Top bar display, download / upload rate — External Downloads section in main view
• channel details view

From that list, Chris Webber is working on a patch for channel details view which I think is close to done. I think I’m going to tackle Preferences next. I’m not sure what other items are spoken for. I’m also not sure how much time it will take to tackle all of these–maybe a month?

That brings me to my next two points:

First, I know it’s taking us a long time to get things done, but it’d be faster if we/I didn’t have to answer issues along the lines of “are you done yet?” and “i tested xyz nightly and the following things don’t work…”.

I really appreciate the fact that you all are eager to help out with Miro development and I really apologize for the frustration that comes with not being able to do anything and having to wait around for a while. We’re working as hard as we can, but it’s just going to take time. I’m in this weird position where I want to maximize the time I’m spending doing the work and minimize the time I’m spending talking about it and managing it, but I don’t want to leave you in the dark about where we are. It’s a tough thing to balance especially where I’m trying to minimize the time I spend thinking about balancing.

Oddly, I bumped into this post on 43 Folders which I think is pretty relevant here.

The second thing is that we’ve had a lot of help from Chris Webber over the last few months. He’s been working in his spare time to come up to speed with the new widget code and help re-implement functionality. His efforts are fantastic, the quality of his work is solid, and he’s absolutely making a big dent in our ability to finish the work faster. You can see the efforts of his work here–I refer to him in the checkin comments as “Chris”, “Christopher” and “Chris Webber”. Thank you Chris!

That’s it for this update!

I merged the changes into the Miro-1.2 branch and cut a tarball. You can get the tarball at http://pculture.org/nightlies/Miro-1.2.2-test.tar.gz.

This code needs testing from distributions that are only using xulrunner 1.9. It “works for me” with Ubuntu Hardy Beta 1 today (but didn’t work yesterday) and it works with Ubuntu Gutsy (where it compiles against Firefox). I haven’t tested it on other distributions.

For the most part, I fixed things that were obvious compile/runtime issues. I didn’t delve into the API differences between xulrunner 1.8 and 1.9 and fix deprecation problems and things of that nature. The changes I made are mediocre, but they seem to work for me. They’re loosely based on changes in the Ubuntu packages. I talked about that in a previous blog entry.

I need help testing this with other distributions. I also need help making sure that no other changes are required. Reply in the comments below, toss a comment in bug 9692, ping me on IRC, and/or send me an email to my pculture.org email address.

The more help and the more eyes we get, the more likely that the code will work where you need it to work.

If no one helps out, then I’ll probably just release it and see what happens.

Note: The changes in the above linked tarball are NOT in the Miro 1.2 or 1.2.1 releases. This is not a final release. This is for testing purposes only.

I continued working on getting Miro 1.2 released. One thing we need to do better is coordinate translations. I’ll be working on fixing that part of the release process for the next release cycle.

I also worked on getting Miro trunk to compile and work with xulrunner 1.9 and on Ubuntu Hardy. I finished this today and checked it in. It seems ok, but definitely needs peer review from someone who’s more familiar with XPCOM.

I had a cup of coffee with Nathan from Creative Commons. He was out here doing work somewhere at MIT and I live nearby so we got together. For Miro 1.0, he implemented license extraction from RSS feeds. You can see the end result of this when you see the “license” item in the feed item details. When the publisher has license information in the feed, this links to the license. When you watch video content on the Internet, think about what roles you play in that content… Are you merely a passive consumer? Can you take the content and remix it into new content with new meaning? Can you share it with your friends? Can you post it elsewhere? The license information answers these questions and more. Thank you Nathan!

He and I talked about other areas where Miro and Creative Commons intersect and then we talked about PyBlosxom–another project I work on.

Once Miro 1.2 is out, I’m going to go back to work on Mediabar. I’ll probably be working on that for the next week or two.

This week was an ok bug-fixing week. I worked on:

• Bug 9644: implemented a preference allowing you to switch between renderers on GTKX11–it’s not great, but it’s good enough (probably).
• Bug 3067: applied a patch for suspending the screensaver when playing video in fullscreen on GTKX11… then I found a couple of problems with it and backed it out today.
• Fixed a bunch of other bugs related to gstreamer renderer, ff/rew, revver, mefeedia, …
• Worked on bug 9214 where if you click on the delete link when viewing a video in fullscreen on GTKX11, then you’re stuck in fullscreen. I worked on it for a couple of days and ended up giving up today.
• Sent out a proposed tentative release schedule for Miro 1.2 to the develop mailing list.
• Helped with some of the initial GSoC preparation.

Next week I’m going to:

• Bug 9692. Check out Fabien’s patches for Ubuntu that get Miro to work with xulrunner 1.9.
• Bug 9691. Look into Miro compiling with gcc 4.3. Uwe said something that suggested that gcc 4.3 on Debian is imminent. So… this needs to be figured out.
• Continue to shepherd Miro 1.2 until it’s out the door.

Busy busy busy….

It was an exciting week:

• Bought a new laptop (Dell 1420n with Ubuntu Gutsy on it, 4 GB of memory and BRIGHT YELLOW)
• Spent a couple of days switching to the new laptop and getting VNC working on my desktop machine
• Worked on bugs 9614, 9627 and 3067–9627 took me a while to figure out
• Centralized the list of projects for internships, GSOC, et al — see it at https://develop.participatoryculture.org/trac/democracy/wiki/CommunityProjects
• Toyed with Hardy Heron alpha 5 for a few hours to get a feel for where we are with that; turns out Miro 1.1.2 is in the Hardy repositories, so that’s a good sign

This coming week I’m planning to:

• Work on 1.2 bugs starting with mine and continuing with Nassar’s
• Take a look at the two-stage problem with the subscribe.getmiro.com site

I want to send HUGE thank yous to the following people:

• mutantmonkey for the patch which disables the x screensaver for GTKX11 when watching movies in fullscreen — this is awesome!
• Njaal for the patch so that Miro will automatically stop seeding torrents when a specified ratio is reached in bug 8793
• Sid for the excellent bug report for 9627
• elmargol for help with bug 9614
• Uwe from Debian and Christopher from Ubuntu for the work they’re doing and the help they give us on packaging

Also, I’d like to apologize for posting my last status so late. I back-dated it, so it probably showed up on the planet pretty low.

It was a pretty slow week.

• I spent some time trying to implement 5403 and 8619 on Mac OSX, but then discovered that Luc must use some gui builder tool that I don’t have. So I ended up passing those two bugs off to him.
• I spent some time doing bug triage.
• I fixed a few problems from code flux.
• I worked on 9521 for a few days. It’s a lot more complicated than I thought it was partially because it touches sections of the code I haven’t touched yet. Nassar thinks it’s probably not something we want to bother with. I think I might be able to implement it, but I think it’ll take a week to get it right. I think we should defer it to a future version.

Things I’m planning on doing this week:

• Possibly continue working on 9521 depending on the call.
• The two-stage installer bug is still out there and I have access to pcf3 now, so I think I can solve it. I sent an email to plans regarding that subscribe.getmiro.com site being all kinds of bit-rotty and whether I should spend time fixing it, but didn’t get any replies.
• We got a patch for bug 8793 for implementing ratio restrictions for torrents for GTKX11. The bulk of the patch is the glade code for GTKX11; the actual code is pretty trivial. I’d like to look into this and probably apply it.
• Check out Miro on Ubuntu Hardy (which is in alpha 4 now) and get a feel for where it’s at.

This actually covers two weeks of work because I was in Florida for a bit in between.

Things I did:

• implemented 5403 and 8619 for GTKX11 and Windows
• worked on Mediabar specs (briefly)
• went to the Worcester office to hang out with the Nicks
• bought a Mac mini for Boston devs
• wrote a blog entry about the Firefox 3 work I did so it’s clearer as to what I was working on and how it impacts Miro
• did a 1.1.2 release which contains the exception text for OpenSSL and GPL co-existence (though it’s not entirely clear if I did this right)
• did some bug-fixing on GTKX11 to get the timebar and video information displaying correctly
• did some re-working of the theme I’m using on my devblog so it’s less annoying–still needs some help, though

Things I’m working on this week:

• finishing up work on bugs 5403 and 8619 (though I may end up giving up–I’m not really following where oid comes from in the strings)
• go through the bugs and do some more light triage work and start marking bugs that look bite-sized as such

I spent the week working on:

• Firefox patches for 303645, 400061, and 400064
• Fixing build issues with Miro 1.1.1 on Gutsy (i386 and amd64), Feisty and Dapper
• Mediabar specs and revival
• Some minor bug triaging
• Bug 9150: implementing feed:// and feeds:// support at the command line

This coming week I’m:

• Working on Mediabar specs and revival
• Any post-landing Firefox patch work
• Working on bugs 9523, 9521 and 5403
• Getting a Mac-mini for Boston devs for testing/development
• Visiting the Worcester office on Thursday, February 7th

It’s been a good week overall. The Firefox 3 B3 codefreeze is tonight and I think I’m on track for landing all three patches, but they haven’t landed yet (as of this writing). We’ll see what happens tonight.

Update: All three patches landed. The one for 400061 had some issues, but myk fixed them and landed it for me.

Things I did this week:

• Visited the Worcestor office on Thursday.
• Cut a 1.1.1 rc1 release.
• Cut a 1.1.1 final release.
• Built Gutsy, Feisty and Dapper packages for 1.1.1… then spent half a day dealing with repository issues resulting from Gutsy i386 miro-data package being slightly different in some way than the Gutsy AMD64 miro-data package. I haven’t figured out the problem here. I’ll try to do it this week. I suspect foul play. Probably my fault. If you’re having problems with Gutsy, Feisty or Dapper packaging for Miro from the PCF repository, let me know.
• Pulled an all-nighter Friday night working on and finishing up patches for 400061 and 400064. Then spent several days talking with Mozilla folks, fixing the patches based on reviews and generally trying to get as much done as possible to increase the chances of these patches making it into Firefox 3.

Things I plan to do this week:

• Continue pushing the patches for 303645, 400061 and 400064 until they’re landed.
• Figure out what the problem with building the miro-data package is.
• Figure out what to do about a new laptop. I’m currently looking at the Dell 1420N and System76 laptops. Any thoughts on Ubuntu-based laptops would be hugely useful.
• Try to avoid getting involved in anyone else’s work until the patches for Firefox are out of my hands.
• Spending some time with my wife before she forgets who I am.

I haven’t put Dapper and Feisty builds for Miro 1.1 into the repository yet. The Gutsy builds are there, but there are some issues with segfaulting when watching videos with them. I’ve only heard about Gutsy segfaulting with Miro 1.1 from one person and there aren’t any new bugs for the issue. From that I’m guessing the issue is pretty limited user-wise, but don’t really have a good way to measure.

The last few days went like this. We did a Miro 1.1 release on Thursday and I started building Ubuntu builds for Dapper, Feisty and Gutsy that afternoon using the new pbuilder-based scripts I’ve been working on. The pbuilder-based scripts are great in that I can automate building packages for Dapper, Feisty and Gutsy for i686 on a single machine (no longer need VMs) and they verify the build-depends lines in the .dsc files. That’ll make building from source possible.

The problem with Miro 1.1 is that the switch from BitTorrent code to libtorrent code causes compiling to take longer. Additionally, the pbuilder-based scripts pull down all the dependencies and build the environment to do the build in for each distribution and that takes a while, too.

When working on builds, I had problems with the Dapper and Feisty builds segfaulting when playing videos during testing. I first blamed the new build scripts. I spent 8 hours or so fiddling with them, verifying all the build steps, and eventually running them in the distribution VMs I had. On Saturday, I decided that theory wasn’t a good one.

I tried a few other things and then started bisecting the svn changes since Miro 1.0 in my Feisty VM to see if I could find the checkin that caused the problem. After a few more hours, I discovered that it was a change to xine_impl.c that I made for bug 9373 that causes the segfaults when viewing videos. Another hour later and I verified this is the same problem with the Dapper build.

I backed out that change and re-ran and re-tested everything.

In summary, the pbuilder-based scripts are fine and backing out that xine_impl.c fix fixes the issues I was seeing.

We’re working on a Miro 1.1.1 release that has some changes that allow for co-branding. We decided to push these changes off to 1.1.1 so that we could release Miro 1.1 a week earlier. I decided that I’d skip builds for Feisty and Dapper for Miro 1.1 and instead do builds when we released Miro 1.1.1 this week. That should happen in the next day or so.

I really apologize for the current situation. It was a confluence of several circumstances that led to me taking a long time to figuring out the cause of the problem which sucked.

I should have 1.1.1 builds of Gutsy, Feisty and Dapper out by Friday night if not sooner.

It was a really busy week–I essentially worked for the last 7 days straight. It’s been a good/bad week for me. I:

• pushed out the Miro 1.1 release on Thursday which had some minor release-engineering issues (I forgot the symlinks for Miro.dmg, the appserial number is 20080101000 but should have been 20080110000, and Feisty and Dapper won’t build)
• I updated the instructions for releases in the wikis which improves the release process going forward
• I spent Thursday night, Friday and most of Saturday investigating the segfaults with the Dapper and Feisty packages and traced it down to the fix for bug 9373
• I backed out the fix for bug 9373–I’ll release Feisty and Dapper packages when we “release” Miro 1.1.1
• I worked on the Firefox patch work for bug 400061
• I made some more changes to the patch for bug 303645, it was reviewed, and it looks like it’s on its way to Firefox 3 (assuming all goes well)

I’m going to spend the next week:

• aiding Paul with the Miro 1.1.1 release
• continuing the work on bug 400061 and friends

I spent the last week:

• fixing some bugs (9373, 8138, 9383, 9372, 9414)
• doing some bug triage
• reworking the Ubuntu packaging scripts to use pbuilder (much easier to build packages and pbuilder verifies the build-depends)
• added a NEEDSINFO resolution to Bugzilla
• tweaked some stuff so we can tag with names like Miro-x.y instead of Democracy-Player-x.y
• mothered through Miro 1.1 rc0 and rc1

I’ll be spending this week:

• finishing up the Miro 1.1 release
• finishing up the Miro 1.1.1 release (with turbo co-branding ass-kicking action)
• working on Mediabar
• fixing some more bugs

I wasn’t around for the last week. I got home last night and spent today recovering from some flu I’ve been fighting for the last week and also continuing work with building a pbuilder environment for Gutsy and friends.

Tomorrow I plan on finishing up the pbuilder stuff and working on the Firefox patches, but … I don’t think I have much of a chance of finishing them in time, so I think I may put it on the backburner and help out with Miro 1.1 and continuing development. Then work on the Firefox patches for 3.1.

Happy New Year!

Figured I’d send this out early because I’m not going to get anything more accomplished.

I spent Wednesday, Thursday and Friday:

• triaging old bugs
• continuing to clean up comment spam left over from when our bug system was managed by Trac
• fixing my Windows build environment problem (bug 9327)
• fixing the problem with Miro on Windows in the 1.0 branch (bug 9363)
• checked in a fix from a downstream reported problem where Miro on GTKX11 doesn’t honor the –xine-driver option (bug 9373, r5897, r5898)
• started making adjustments to our build scripts to allow us to tag with “Miro-1.1″ instead of “Democracy-Player-With-White-Wine-Sauce-1.1″ which is too long and I’m sick of typing it
• set up a pbuilder environment for testing Ubuntu/Debian packages and continuing to make our packages better

That about covers the last three days. pbuilder is really cool, but takes forever to run because it’s building a fresh environment in which ti build everything and it does that by downloading and installing all the packages each iteration. It’s definitely a good idea to do this, though, as it’ll increase the confidence in our Ubuntu/Debian packaging and potentially fix outstanding issues.

I’m doing a whirlwind tour of family visiting over the next few days, then going down to Louisiana on a service trip to rebuild houses. I’ll be back online January 1st.

I submitted a new patch for bug 303645 on the 12th. On the 13th, I started having problems seeing the feed preview page. I talked with the folks on #develop, but wasn’t able to get it working. No clue what the problem is. I did a fresh checkout on a different machine and that works fine, so I continued working on the second machine and produced a better version of the 303645 patch. I’m gone next week and between that and the time I’m going to have to put into the 1.1 release, I don’t think I will get a patch together for the rest of the changes for bug 400059.

Firefox takes me about an hour to compile from scratch. While trying to figure out what my feed preview page problem was, I spent some time using the reports feature in Bugzilla and doing bug triage on old bugs. I also wrote a script to remove comment spam from the comments we migrated from our old Trac bug-tracker. There’s still a lot of comment spam in there. Ick.

I merged the libtorrent changes from trunk to the 1.0 branch and tested out the GTKX11 platform. I went through Ubuntu Gutsy, Gutsy-64, Feisty and Dapper, made sure Miro in the 1.0 branch compiled and updated the requirements as listed in GTKX11BuildDocs.

I tried to test the 1.0 branch of Miro in Windows, but I’m hitting the LIBEAY32.dll problem in bug 9327 and the fix suggested in the comments isn’t working for me. So… my Windows build environment isn’t working again and I don’t know whether the 1.0 branch is stable or not after the merges I did.

That about covers me for this last week.

In the next few days, I plan to help out with the 1.1 release, make sure I have the Ubuntu platforms covered and continue working on the Firefox patch.

I’m gone from December 23rd through December 31st. After December 22nd, I won’t be online again until January 1st.

It’s been a good-ish week.

Nathan finished up his changes for bug 9077, they were merged into the trunk and 1.0 branches, and Janet did some testing and came up with bugs 9334 and 9335. I fixed 9334 but I think I’m going to skip 9335–it involves changing the padding for those items in the item view and I’m not wildly excited about doing that and I think the issue is cosmetic. We only implemented CC metadata at the item scope–not the feed scope, yet.

I worked my way through some bonehead issues I had caused, finished up the patch for bug 303645 and submitted it. I’m a little apprehensive about submitting a patch to Firefox, but … I’ll suck it up. The important thing is that this patch populates the enclosures array for each FeedEntry item. That was a pre-requisite for bug 400059. I’m working on that one now. As a side note, the folks on #developer on the Mozilla IRC channel have been really helpful.

On Friday, I went to lunch with Dean, Chris Blizzard and John Resig–that was really neat. A little hard to quell the star-struck feelings–hopefully I didn’t make a total ass of myself.

I created the “other packages” page for the download section of the getmiro web-site and changed around the download page, too.

I sent an email to Justin at Mozilla in response to his blog entry. I pointed him to the code for the timeline script and the script for migrating data from Trac to Bugzilla.

Looking forward to a Miro 1.1 release with all its libtorrent and CreativeCommons metadata goodness….

I’ve been super busy this past week.

I spent the brunt of my time on the Firefox patch. I’ve solved most of bug 303645, though the output isn’t pretty and it doesn’t support Yahoo’s MRSS or iTunes enclosures. I think I’ll have it figured out in the next couple of days and submitted to the Mozilla folks. Once I’ve populated the enclosures and they’re available in the FeedWriter, I can work on what we really want to do. I still need to figure out how to associate applications with different feed types. Mozilla froze the trunk today for the upcoming beta 2, so none of the changes I’ve done will be available until beta 3 at the earliest.

I spent several hours looking for enhancements or bugs that lend themselves to being small 1-5 day tasks for high school/college level people in the PSF section of the GHOP. I didn’t find any that I thought were promising. This is a bit unfortunate as it’d give us some good exposure, gets us some help, and would be good for the project. Still, there’s a certain amount of work that would need to be done to be part of the GHOP. The contest lasts until February, so I’m hoping I can figure something out before the end.

On Friday, Dean and I talked with Henri of CivicActions. They work with clients who produce content and we talked about various directions our groups can take to help each other. He’s particularly interested in how Miro could interact with mobile and embedded devices. I’m pretty interested in that, too. I plan on working on that when I get a Nokia n810. I’ve written about that in previous posts.

Also, I’ve been working with Nathan of Creative Commons to get Miro to understand and work with licensing metadata (bug 9077). He’s done most of the work so far; I’ve been providing feedback and working out the implementation issues. This is really useful since it allows content producers to embed licensing data in the feeds that Miro will display to users viewing the content.

And I’ve spent some time doing bug triage and talking with users about various issues, mostly related to packaging.

Current deadlines:

The CreativeCommons birthday is 12/15, so we need to have the 9077 work done by then.

The Mozilla folks told us that we need to get the patch done before the end of the month.

We were thinking of doing a Miro 1.1 release mid-month. I want to fix the packaging scripts so that we can name tags and branches Miro-x.y instead of Democracy-Player-x.y. I’ll probably look into that later this week.

Short status this week….

I took Wednesday through Saturday off.

On Monday, I fiddled with my Windows build environment and finally got it working (again) on Tuesday. I was having problems getting fasttypes to compile. I was getting all kinds of errors when going through the boost stuff. After some skulking through Google results regarding boost compilation problems, I decided to try installing Visual Studio 7.1 without installing the service pack. That worked super–though I’m not entirely sure why. I updated the WindowsBuildDocs page with new urls and tried to break up the instructions into something that’s more digestible.

I also continued to work on the Firefox patch.

I spent Wednesday, Thursday, Friday and part of Saturday watching new bugs, helping users out with 1.0 issues, continuing to build a Windows VM (I’ve almost got it working again) and working on Mediabar.

I checked in a minor overhaul of Mediabar. There are two big issues with Mediabar that need to be fixed that involve architecture changes (I’ve been talking about this for a month now). I figured since I’m overhauling the code for that, I might as well overhaul the code and fix namespace issues and tighten up the existing architecture to make it easier to fix the big issues. In the process of making the changes, I noticed the flv extraction code doesn’t work. I’m not sure how it’s supposed to do what it does, though, so I’m not sure if it’s something I did or something that was pre-existing or something I’m misunderstanding. When I work on Mediabar again, I’ll talk to NPR and Dean about how it should behave and what kinds of things it should be picking up and write it down into an ad hoc specification. On a side note, anyone have any idea how to do agile-like development with Firefox extensions? Where does the testing code go and how do you kick it off?

On Sunday, I got worried that I’m going to miss the deadline for the Firefox patch I’m working on. The work is under bug #400059 in the Mozilla Bugzilla db. I spent Monday and Tuesday working on adding enclosure detection to the FeedProcessor and then adding enclosure support to FeedWriter so that you can see enclosure links on the feed subscribe preview page. When I get this working, I’ll submit it as a patch against bug #303645. Making those changes paves the way towards adding support for distinguishing between video, audio and text feeds and supporting applications for handling those different feed types.

I will be off of email and IRC for the rest of the week but I’ll be studying.

I hope you all have pleasant holidays or work days (depending on where you live)!