Larry Osterman's WebLog

> I'm not sure if public bug databases can scale to a project the size of Windows.

... what's the limiting factor?  Screening of user input?

The question is how do you compare cost (majority of which is hard cash) to benefits which are mostly intagible. I think anyone today can already submit a bug, even though there isn't a clear way to do it, so screening user input is already being done (to what degree that screening consists of simply ignoring the input is another question). So this comes down to making the bug database publicly readable. I can hardly see any harm in that, if anything it will put more presure on the team to fix things.

Everyone needs to blow off steam sooner or later... Whether it happens in a public bugs database, or a blog somewhere, chances are it'll still show up in the Googe results for "IE sux" somewhere. :rolleyes:

Don't forget about communications outside of Microsoft's area of management. Every dev forum and blog would locked up solid for months, even years, just trying to assimilate all that "new" information!

I would recommend two databases. One public. One private.
I've been involved in both public, and private field tests for software development projects and the feed back received via private bug reports has been predominantly useful.

In my experience, when you analyse public feedback, you see that ~70% of the reported problems are due to "New features" that actually require "New" users to read the "What's new" help pages.

BTW, how's the re-org going :-)

Stewart

> I think anyone today can already submit a bug, even though there isn't a clear way to do it

I had to call PSS (at a cost of $99) to submit a bug in IIS 5.
After PSS confirmed it was a bug, I got the $99 refunded.
(Alas for me, the bug was marked WONTFIX... see http://blogs.msdn.com/david.wang/archive/2006/03/19/On_IIS_Team_Blogging_Part_2.aspx)

The $99 is a very good screening mechanism.  If someone hates Windows sooooo much that they're willing to pay $99 to rant, I'm sure Microsoft would be willing to take that money.

> we have private databases

So the question then becomes whether cost or value is added (in net) by maintaining seperate solutions for private bug databases and public feedback mechanisms... as opposed to a few extra fields (IsPrivate as bit) on a common database infrastructure.

Assuming that a common database infrastructure makes sense (which it doesn't, necessarily) then the question would become whether that advantage would surmount the migration costs.

I think the scale question comes down to whether it is easy for someone who has also noticed the problem to find an existing report.  That does strike me as challenging.  But the recipients of all of those reports have to deal with it too, so there must be a way.

I'm in favor of an on-line bug-report and tracking system for one simple reason.  It cheeses me off to go to a lot of trouble to document and submit a bug that the producer already has reports about or knows about.  The other irksome part is having no idea what the disposition is, when there's a disposition.  So to submit bugs, I have to be willing to be expending lots of energy that may be a waste of time.  I don't like that and it moderates my willngness to bother to submit.

I notice I have the same problem with reporting security problems (not to secure@microsoft.com though) and phish e-mails to the security sites of various organizations.   All I get is the gratuitous form letter and no clue that I didn't just waste my time.  I notice that new phish have been accumulating in my quarantine and I should just start deleting the suckers if I don't report them right away.

I am forced to disagree with the idea of a public bug database.  The problem would be all the 15 year old slashdotters and diggers who would flood the thing and try to hack it.  

Look what happened when Mass. made public the debate about file formats.  All the open source zealots voted up for a file format that hardly anyone uses at the risk of creating all kinds of expense just so they could get one over on MS.

James

I'm a big proponent of groups owning responding to public bugs for their component. There's nothing like having to deal with the pain that actual users are going through with your stuff to change your perspective.

EricGu

IMHO the $99 support call to file a bug is too high of a hurdle.  I've found bugs in MS products before (one in IE's DOM just today) and would gladly report it if it were easy.  I've got no incentive to report it if I have to pay though.  I'm already offering my (employer's) time, don't make me get a PO too!  I was able to find a workaround, and since it's code for a web app, even if MS fixes it, I still have to use the workaround I found until everyone is on IE 7.  So this thing will probably remain broken, when I know that the engineers would appreciate bug reports.

>> The $99 is a very good screening mechanism.  If someone hates Windows sooooo much that they're willing to pay $99 to rant, I'm sure Microsoft would be willing to take that money.

Wow, next time I will steal money from other people I will surely call it a screening mechanism!

Perhaps I'm the most bitter poster of public bugs in the public bug databases operated by you and Raymond Chen, but obviously I'm not the only poster.  It seems most such bugs have a permanent attribute of "wontfix".

How is it that open source products can afford to have public bug databases operated in a more formal manner?  They often have more bugs than Windows-related products (at least when they start).  How do they survive with their bug databses?

Open source products typically have a couple orders of magnitude fewer people using them than Windows does so the problem of sheer volume isn't as big.

Also since Windows is the current top dog it's the one that everybody wants to take down.  You don't get any cred for hacking JoeBoy's app that noone ever heard about.  You get lots for shutting down Window's public bug database.

Having said that it would be nice if there was more open flow between MS devs and thier users.  I don't know what the right model is though that allows this flow while at the same time doesn't completely randomize the devs from getting thier jobs done.

There is a public bug database for most developer products

http://lab.msdn.microsoft.com/productfeedback

It also has the community involved by voting

I don't see a big problem with making entering bugs a hurdle, it will cut down on trolls. Charging people $99 to report a bug will not eliminate all of them but it will cut down their numbers significantly. The real benefit comes from the fact that everybody will be able to verify whether what they found really is a bug. If I have an issue and I can verify that in a public bug database I will not have to waste my time (and money) trying to fix something that is not an error on my end (and reporting a bug that's already been reported). Eventually the database will also be filled with common misconceptions about how things should work and all those WONTFIX or By Design bugs will simply include comments on why those issues are not bugs. That will help people stop reporting these issues and even better, help them understand the logic and change their use of such feature to the correct one, not what they think it should be.

>Perhaps I'm the most bitter poster of public bugs in the public
>bug databases operated by you and Raymond Chen, but
>obviously I'm not the only poster.  It seems most such bugs
>have a permanent attribute of "wontfix".

Thats probably because the bugs you keep on posting about usually relate to data you lost on a scsi drive attached to a laptop running windows 95.  

In the past (e.g., my distant mainframe days), transparency about bugs was moderated by a concern for liability.  That may still be a factor for many.  

It was startling to me, years ago, when I learned that Burroughs operated a public bug database and you could find it and submit to it on-line (e.g., via teletype dial-in).

With regard to being drowned by the trolls, this strikes me as about worrying over a problem before it happens.  It takes a bit more to file a bug report than simply be snarky, and requiring registration and some degree of non-anonymity seems perfectly appropriate.  I figure SourceForge bug lists don't get bombed because there is no charge in it (e.g., pent-up anti-Microsoft rage).

I like the MSDN lab approach and look forward to it becoming the standard for doing business.  There will need to be more work on organization for complex products, obviously.  And it is fun to see a report in Japanese being responded to in English.  Welcome to the global world of software development.

Friday, March 31, 2006 8:48 AM by dhiren
> Thats probably because the bugs you keep on posting about
> usually relate to data you lost on a scsi drive attached to a
> laptop running windows 95.

Got it.  Perhaps that's the reason why I'm the most bitter poster of public bugs on the blogs of Messrs Chen and Osterman.  Perhaps that's the reason why I'm not the only poster.  Perhaps lots of others have posted lots of less serious bugs because I'm the one with the most bitter experience.  Could be true.  Partly.

I didn't lose the data.  I lost entire contents of partitions.  In Windows 2000 I also lost entire contents of partitions.  In Windows XP SP2, Windows Server 2003 SP1, and a certain beta which doesn't count because it's a beta but it does need to be fixed, I only lost a few instances of files.  Windows 95 taught me to keep 6 copies of important files and 3 copies of most other files.  Windows XP temporarily taught me to be lazy, but not any more.  I don't lose many files.  When I know which instances are lost, I recover them from other copies.

I did lose a lot of time over a few months tracking down some of the problems, and around 11,000 yen spent on train fares going to hardware manufacturers and going back to the retail store where I bought the stuff, none of whom could deliver fixes for Microsoft's work.  I learned what percentage of truth there was in the asserted 90-day warranty that came with Windows.

Five minutes ago I found a new (not new but first I've seen of it) in which Windows Explorer bug in Windows XP SP2 overwrites files which it should not overwrite and it lies about the filenames involved.  It is 100% reproducible.

I do not offer to pay Microsoft's bug reporting fee even if Microsoft decides to offer a refund after figuring out that the bug is a bug.

By coincidence the situation in which I discovered it was that I had reported an MSDN bug to Microsoft and the e-mail had a long subject line.  This provides a clue about how to start on reproducing the newly discovered Windows Explorer bug.  Of course full details do need to be posted, and publicly.  I will be happy to post full details if it's acceptable.  Mr. Osterman, will you be kind enough to let your blog become more useful as a public bug database?  Or persuade your employer to operate a proper bug database?  Or should there continue to be no ethically responsible way to report this kind of stuff?

"I didn't lose the data.  I lost entire contents of partitions.  In Windows 2000 I also lost entire contents of partitions.  In Windows XP SP2, Windows Server 2003 SP1, and a certain beta which doesn't count because it's a beta but it does need to be fixed, I only lost a few instances of files."

Are you sure this wasn't due to harddrive failure / pending failure?

Monday, April 24, 2006 4:25 AM by n4cer
> Are you sure this wasn't due to harddrive failure / pending
> failure?

Yes.

Originally with Windows 95 I wasn't sure.  At the retail store where I'd bought all the stuff, store employees and I tried two other makers' external SCSI hard disks (changing new merchandise into used merchandise) and results were identical.  We tried one other maker's PCMCIA-SCSI card (again changing new merchandise into used merchandise) and results were identical.  The PC's maker tried a different PCMCIA-SCSI card that they had for testing, and results were identical.  The maker of one of the PCMCIA-SCSI cards tried different models of cards that they had for testing, and tried different hard disks including different capacities, and results were identical.  After several months and about 11,000 yen in train fares going to the store and vendors, I learned how to reproduce it in 10 minutes.  If you'd like to visit me we can reproduce it in 10 minutes.

With Windows 2000, my first episode occured with a hard disk that had been used internally in a notebook running Windows 98, which I put in an external case with PCMCIA-IDE adapter.  Ordinarily no errors occured.  But one time I connected the drive before booting Windows 2000.  Windows 2000 early in its boot sequence displayed an assertion that the drive needed to be checked for consistency, I allowed it to do so, and it deleted a ton of files from the partition.  The result had a ton of files remaining but was still pretty useless, so I reformatted the partition and copied all the files back in from other backups.

With Windows 2000, my second episode was with a USB hard disk (made that way by the manufacturer i.e. not adapted by me).  Same thing again, if I made the mistake of connecting it before booting Windows 2000, then Windows 2000 destroyed the contents during booting.

Windows 2003 tried to do the same thing to me, but I caught it just in time.  Something like "Press any key within 10 seconds in order to prevent destruction, oops I mean in order to prevent checking".  After Windows 2003 booted I ran a checkdisk.  No problem with it after Windows 2003 was up and running.

Then there are the cases you quoted me on with Windows XP SP2, Windows Server 2003 SP1, and one more.  In these OSes, if a hard drive has a bad block that causes read errors, Windows records an event in the system log saying that the drive has a bad block.  But if the drive doesn't have a bad block, and if Windows discovers that Windows wrote corrupt information in the file structures for the partition, then the event logs are different.  Fortunately Windows is pretty much honest in these cases, and it is clear where the blame lies.

Then there's the Vista beta.  This wasn't even an installation on a real machine, it was in a virtual machine under Microsoft Virtual PC 2004.  Vista thought it was accessing an ordinary IDE hard drive, but all of that was really just in one file on the internal IDE drive of a physical machine running Windows 2003 SP1.  If the physical drive had a bad block then the real Windows 2003 SP1 would record an event in its system log to say so.  That did not happen.  The guest machine running Vista also didn't have a log entry about a bad block, because that's not what happened.  The guest machine running Vista had a log entry saying that the file structures in its partition were corrupt.  This is the same kind of corruption as XP SP2 and 2003 SP1 had done on physical USB disks, but with twice the amount of proof.

So yes, I am sure.

By the way I have also had about four physical hard drives that did have bad blocks, and still have two with uncorrectable bad blocks.  It's pretty trivial now to conduct a comparison when necessary.

Windows causes more destruction than actual bad blocks do, by many orders of magnitude.