Smolt, for make benefit glorious QA

by davidnielsen

I’ve been poking Ken VanDine for a while to get Foresight Linux behind smolt. In the progress we started talking about cool things to do with Smolt, aside getting good metrics on users hardware that is. One thing captured the coversation though, using smolt to improve QA. The idea was basically to hook smolt up as the backend to get metrics on the hardware, then have each machine tracked (sparing the privacy parts naturally), you could then hook up something like Ubuntu’s hardware tester so that the user feedback on the status of hardware the requires interactive feedback such as does the soundcard work. The desire then would be to make a certain list of packages like the kernel hook up to this framework so that when an update is issued and installed, the user would then be kindly asked to check if everything still works. I imagine other packages like pulseaudio would also benefit from such basic testing. When the user then tells the tester (or it discovers this automatically – the ideal case) that a piece of hardware that used to work but no longer does, using the magic of xml-rpc a bugreport could be opened automatically informing the developer in question of the regression, all relevant information could hopefully be extracted using Apport and our good friend HAL.

Another thing we’d get for free with smolt would be a database of hardware that is known to work with Linux and we can even say which versions of a given distribution will work and show a rough level of stability. All of which should prove valuable for such people as Average Joe and the nice OEMs such as Dell. Not to mention this would likely be the first attempt at creating such a database that would actually succeed.

Smolt could also provide knowledge of problematic combinations of hardware, something that might just prove useful in tracking down problems. Not to mention we could track which machines succesfully suspend and resume, a considerable problem area – none of my machine e.g. do this, it might be nice for such a system to prevent users from suspending if the combination of hardware is reported as not working or at least warn them. It would have saved me some time at least.

Then we integrate this sort of testing with the installer, and voila, we know if the hardware is supported and can warn the user before he nukes his data and then yells at us over the disappointing failure.

Just a rant, likely the result of considerable injestion of caffiene, but I do think in the larger picture that smolt could get to play a valuable part in getting use much neede metrics and as a tool to improve the platform. Thus I’m happy to see the developers work to get it accepted with other distributions.