Test Benchmark: Adapt tests to modern architectures

Test::Benchmark: Adapt tests to modern architectures

Recently I tried to install Test::Benchmark from CPAN via 'cpanm'. It failed to install, which was surprising, because code written by its author, Fergal Daly, is usually very sound.

I went to the cpanm build directory and manually ran 'perl Makefile.PL; make; make test'. I got these failures:

[Test-Benchmark-0.004] 48 $ make test
PERL_DL_NONLAZY=1 "/home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/test.t .. 13/36
#   Failed test 'subtest '10 faster than 20 num' of '10 faster than 20' compare actual_ok'
#   at t/test.t line 60.
#          got: '0'
#     expected: '1'

#   Failed test 'subtest '10 faster than 20 num' of '10 faster than 20' compare diag'
#   at t/test.t line 60.
# Got diag (141 bytes):
# 'code1 was not faster than code 2'
# ' 0 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)'
# ' 0 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)'
# Expected diag (0 bytes):
#
t/test.t .. 24/36 # Looks like you failed 2 tests of 36.
t/test.t .. Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/36 subtests

Test Summary Report
-------------------
t/test.t (Wstat: 512 Tests: 36 Failed: 2)
  Failed tests:  20, 23
  Non-zero exit status: 2
Files=1, Tests=36, 18 wallclock secs ( 0.02 usr  0.00 sys + 17.26 cusr  0.44 csys = 17.72 CPU)
Result: FAIL
Failed 1/1 test programs. 2/36 subtests failed.
make: *** [test_dynamic] Error 2

I then went to the bug tracker for Test-Benchmark (see link below). I found that this bug was reported nearly three years ago. The module has been failing some, but not all of its CPAN testers' tests as well (link below).

It appears that the code is sound but that the tests are written on the expectation that certain benchmarks will be faster than others. That was a reasonable expectation back in the 32-bit days, but no longer so on 64-bit architectures.

Project Request

Determine what kind of tests are needed to test Test-Benchmark's functionality regardless of -- or perhaps conditionally upon -- architecture. Write a patch and submit it to existing bug report.

Resources

• Links

  • Source Code: http://search.cpan.org/~fdaly/Test-Benchmark-0.004/

  • Bug Report: https://rt.cpan.org/Public/Bug/Display.html?id=78290

  • CPAN testers results: http://matrix.cpantesters.org/?dist=Test-Benchmark

  • Source Code Repository: None listed; module last updated in pre-github.com days.

• Version Control System: N/A.

Tasks

• Develop a good understanding of what Test::Benchmark is trying to accomplish.

• Determine how Test::Benchmark is currently being tested. This will require reference to Test::Tester.

• Determine why some of Test::Benchmark's current tests are failing on certain architectures and not others.

• Write (or re-write) tests that will either work as expected on any architecture or that test for architecture before being run.

• Submit patch to author by attaching to bug report listed below.

Participants

[Become the first participant in this project!]

Created by: Jim Keenan. [email protected]. irc.perl.org: kid51.

Track

CPAN Authors Requests