Help with Launchpad Bug Import Script

I and my fellow Launchpad bugs hackers have always helped people migrate to Launchpad by doing bug imports for people running other bug trackers. This has increasingly taken more and more of our time, and we've gotten worse at doing it in a timely manner for people. This is mostly because we really want to get fixes and improvements to the bug tracker itself landed. And really there's no need for us to do all this. Anyone can import their bugs into Launchpad using a Launchpad API script.  So here's where I come to you, dear reader, asking for help....

Does anyone have an existing API script to import bugs into Launchpad they would be willing to open source? Or would anyone be willing to take our existing Launchpad import script and convert it to use the API? Ping me in the comments on this blog or via email if you're interested. Or catch me on IRC on the Freenode network (nick is deryck).

For those who might be interested in converting the current script, take a look at the existing Launchpad bug import script. Note that this script assumes you've created an import file in Launchpad's bug import format.

Posted by deryck on September 1, 2010


Gavin Panella on September 3, 2010 at 5:32 a.m.

The bug importer does special, super-user-like things. For example, it adds bug comments with the sender and date taken from the original comment on the bug tracker it's importing from. Hence, without some big changes, I think that Launchpad can't support bug imports via the web API.

However, a lot of the existing process can probably be automated. For example, Launchpad could directly allow bug import submissions, and tools could be released to help people check the validity of their bug import data (and this could probably use the API).

deryck on September 3, 2010 at 8:09 a.m.

Ah, that's a good point, I hadn't even thought of the super-user-ness of the current script. Kind of dumb of me. So that completely kills my "use a script" plan. The alternative is soooo much more work, though.

I wonder if we could allow some mini-superuser powers via the API? For bug supervisors, for example. Allowing them to set dates on bugs and comments, reporter and message owner. Perhaps that's madness. :-)

Oh, well, bug import duties are with us a bit longer, I see.

Gavin Panella on September 3, 2010 at 9:45 a.m.

Yeah, I think so :-/

From what I remember, the biggest pain points are:

- Cleaning up the import data. In the past I've written multi-stage pipelines to fix broken import XML because it's more efficient (globally, maybe not for me) than going back to the requester.

- Getting the bug import script run, because of LOSA availability, especially because getting an import done can take several iterations.

The first point we can help by creating a way to pre-validate bug import XML. Perhaps that could take the form of a downloadable script or a service Launchpad provides.

The second point we could alleviate with a form to upload the import data and a job runner to process it. The job should also store its retry information in Launchpad (rather than in a pickle!), or, for small imports, do it in a single all-or-nothing transaction.

I think those two could drastically reduce the workload of the Bugs team, and equally improve our responsiveness to import requests.

Post a comment