[racket-dev] Migrating the bug database to GitHub

Summary: I'm preparing to switch the Racket bug database to GitHub.
At the end of this email, there's a long list of reasons why (and a
few limitations).

Action: Please visit http://samth2.ccs.neu.edu:8080/github-auth/ to make
it possible to associate your existing bugs with your GitHub account.

=======

Going forward: we're going to switch from maintaining our own bug system
to using the Issues database provided by GitHub.  All existing bugs
(except "bogus" bugs) will be migrated, with all of their data.  We will
keep most of the features we currently have, as well.

Here's how it will work in the new system:

1. You'll be able to enter a GitHub username and password in DrRacket to
   post bugs as your GitHub user.  For people who don't do this (mostly
   for people who don't have a GitHub account), the bug will be
   submitted using a GitHub account that is specifically set up for this
   purpose.  In the latter case, the bug will show up on the GitHub
   system as coming from that account, but the bug text will have the
   original email for notifications.

2. If you use a GitHub account to create a bug, notifications for
   changes to the bug will be sent to you as usual from GitHub.  For
   non-GitHub bugs, a script that gets bug change notifications will
   send out emails to the original email that was used to submit the
   bug.

3. If you currently have an account that lets you edit the Racket bug
   database (you should already know if this applies to you), you will
   need to setup a GitHub account if you don't already have one.  You
   will be able to edit bugs from the GitHub web interface, just as you
   could from the Gnats web interface.  You will automatically be
   notified of new bugs, and you can control this via GitHub.

Action: get a GitHub account if you don't have one already, and send the
account name to me.

That's it!  It's really pretty simple.

=======

Migration:  The hard part in all of this is migrating the existing
bugs.  Fortunately, with a bunch of help from Eli, I've managed to
parse the entire existing Gnats database, and use it convert the
existing bugs to the data formats that the GitHub API expects.

You can see samples of the results at the following bugs:

https://github.com/samth/test-bugs/issues/170
https://github.com/samth/test-bugs/issues/171
https://github.com/samth/test-bugs/issues/172
https://github.com/samth/test-bugs/issues/173
https://github.com/samth/test-bugs/issues/174
https://github.com/samth/test-bugs/issues/175
https://github.com/samth/test-bugs/issues/176
https://github.com/samth/test-bugs/issues/177
https://github.com/samth/test-bugs/issues/178
https://github.com/samth/test-bugs/issues/179
https://github.com/samth/test-bugs/issues/180

As you can see, some of the parts of the bugs are formatted in
fixed-width font, as for code, and some not.  In general, for open
bugs I made all these decisions manually, and for closed bugs I used
heuristics.  I may be able to improve these heuristics before I do the
final migration.

However, as you can see, all of these bugs as submitted by the same
GitHub user.  To fix this, I need to be able to link your email
address and your GitHub username, and to be able to post bugs as under
your name.

Action:  If you want bugs you have submitted or commented on to be
associated on GitHub with your account there, then please visit:

  http://samth2.ccs.neu.edu:8080/github-auth/

and follow the directions there.  Please include *all* the email
addresses you've used to submit bugs, including really old ones.
Don't worry if you miss some, though -- I'll do my best to figure out
the missing ones, and you can visit the site as many times as you
want.

======

I'd like to thank Eli for his invaluable assistance with all of this.

Feedback and suggestions on any aspect of this is encouraged.

=======

What do we get from using GitHub issues?

  * Enabling community interaction.  GitHub is where the developer
    community is, and switching to GitHub makes it much easier for
    other people to contribute.

  * A working, maintained system.

  * Simple, modern web interface.

  * Simple label-based interaction with bugs.

  * Various GitHub-related tools that exist now, and will be made in the
    future.

  * Commit-based bug closing, similar to what we have with Gnats.  See
    https://github.com/blog/831-issues-2-0-the-next-generation for
    details.

  * References to bug numbers are highlighted in commit messages when
    viewed via the github interface.

What do we lose from the switch?

  * Complete control over the aspect of the bug system.  (But losing
    this makes the first advantage above.)

  * The bug notification-turned-mailing-list interaction aspect of Gnats
    is mostly lost.  GitHub sends out notifications when a bug is
    created, but it will *not* send out further comments unless you're
    participating in the discussion (i.e., you sent a comment on the
    issue).

    - This makes for the most major workflow change: instead of relying
      on your mailbox for all bug-related activity, you *will* need to
      use the GitHub (web) interface.

    - Note: it is possible to set up the listening script to send out
      notifications for all updates.  However, doing this means that you
      will get these notifications in addition to the GitHub emails,
      making it impractical.

    - GitHub notifications are reply-able: such replies are properly
      recorded as comments on the bug.  However, attachments are
      currently *discarded* from these replies.

  * Notifications sent out by the changes-listening script will send
    their reply to a special email address, which I will have to
    monitor and copy manually to the bug site.

  * As of yet, there is still *no* support for attachments.  Screenshots
    etc will need to happen off-line.  Gnats had bad support for
    attachments which was practically never used, but we have used
    emails for such cases.  This is therefore a lost feature.  (Note
    that there is no rejection of attachments on the GitHub side, it
    might be added, eventually.)


-- 
sam th
samth at ccs.neu.edu