Cem Kaner, a professor at the Florida Institute of Technology, has done research on the ratio of software testers to software developers. His presentation entitled “Managing the Proportion of Testers to Other Developers” is partially based on a meeting of the Software Test Managers Roundtable (STMR 3) in Fall 2001.
The study found that:
– There were very small ratios (1-to-7 and less) and very
large ratios (5-to-1).
– Some of each worked and some of each failed.
– Many remembered successful projects with ratios lower than 1-to-1 more favorably than successful projects with larger ratios.
Read the paper to find out why is there such a range of successful ratios, and why test managers be happy with relatively low ratios?
See: http://www.kaner.com/pdfs/pnsqc_ratios.pdf and http://www.kaner.com/
Read Full Post »
Avishay Traeger from the IBM Haifa Research Lab and Erez Zadok from Stony Brook University are raising awareness of issues relating to proper benchmarking practices of file and storage systems. They hope that with greater awareness, standards will be raised, and more rigorous and scientific evaluations will be performed and published.
In May 2008 they published a paper in the ACM Transactions on Storage entitled “A Nine Year Study of File System and Storage Benchmarking'” in which they surveyed 415 file system and storage benchmarks from 106 papers that were published in four highly-regarded conferences (SOSP, OSDI, USENIX, and FAST) between 1999 and 2007. They found that most popular benchmarks are flawed, and many research papers used poor benchmarking practices and did not provide a clear indication of the system’s true performance. They have provided a set of guidelines that they hope will improve future performance evaluations. An updated version of the guidelines is available.
Traeger and Zadok have also set up a mailing list for information on future events, as well as discussions. More information can be found on their File and Storage System Benchmarking Portal
Read Full Post »