[racket-dev] DrDr Feature Request

Could DrDr say "This build is not the latest" or "The latest push is
234234"?

On 08/08/2011 11:37 AM, Jay McCarthy wrote:
> It is useful to test all of them to find out when errors start. It
> doesn't do the newest first, because then the calculation of "new
> issue" wouldn't make any sense, because you wouldn't have the previous
> push's tests.
>
> Jay
>
> On Mon, Aug 8, 2011 at 11:34 AM, Jon Rafkind <rafkind at cs.utah.edu> wrote:
>> Another request: could DrDr process the latest push first? Its a little
>> annoying to get emails for tests that failed when the latest push fixes
>> them but DrDr is so far behind. Is there any benefit to testing all the
>> intermediate pushes?
>>
>> On 08/08/2011 09:56 AM, Vincent St-Amour wrote:
>>> I love DrDr, but there's a small thing that annoys me about it.
>>>
>>> Some tests are prone to intermittent failures. For example, some
>>> benchmarks need to create a file, and several benchmarks share the
>>> same file, which leads to race conditions. Similarly, some DrRacket
>>> tests sometimes fail for focus reasons.
>>>
>>> So, whenever someone pushes, they may get failures from these tests,
>>> then have go look at the actual errors, and try to figure out if they
>>> actually broke something or not.
>>>
>>> (Or, they ignore these failures, which is bad.)
>>>
>>> Here are two potential solutions. Let's assume that I just pushed
>>> something, and a test started failing.
>>>
>>> - Have DrDr send me email for every push about the broken test for as
>>>   long as it fails. If I get email more than once, it's likely that I
>>>   actually broke something. If I only get email once, the problem went
>>>   away on its own, and was likely an intermittent failure.
>>>
>>> - Have the possiblity to flag some tests as intermittent (something
>>>   like `drdr:random'), and only report failures for these tests if
>>>   they fail twice in a row. This would reduce the amount of noise,
>>>   since I expect most of these tests to pass most of the time. Actual
>>>   breakage would still be detected, since it's unlikely that such
>>>   failures would go away on their own. Detection would happen one push
>>>   late, but that shouldn't be too much of an issue.
>>>
>>>   Or, maybe only notify the pusher after two failures in a row, but
>>>   notify the responsible person right away.
>>>
>>> Any thoughts?
>>>
>>> Vincent
>>> _________________________________________________
>>>   For list-related administrative tasks:
>>>   http://lists.racket-lang.org/listinfo/dev
>> _________________________________________________
>>  For list-related administrative tasks:
>>  http://lists.racket-lang.org/listinfo/dev
>>
>
>