Ninety gigs, down the toilet

Clearly, I know what I'm talking about here.

Yes, I am pleased to learn of Hitachi's plan to release probably-functional "one terabyte" hard drives Real Soon Now. They'll probably work fine, the price is good, it's like Bigfoot or Jesus. Huzzah.

This is, however, a good time to mention that now that consumer hard drives are nudging the "1Tb barrier", the capacity rip-off factor is about to become worse by a factor of 1.024. Again.

As I and others have written many times before, storage manufacturers are, almost without exception, in love with specifying their devices as if a kilobyte is 1000 bytes, a megabyte 1000 kilobytes, a gigabyte 1000 megabytes, and (now) a terabyte 1000 gigabytes.

According to the standard SI prefixes, this is exactly true. There are one thousand grams in a kilogram, after all.

In computer usage, though, those SI prefixes are perverted to refer to powers of two, not ten, despite the so-far-unsuccessful effort of the standards organisations to get everybody to call the computer capacities "kibibyte", "mebibyte" and so on.

So a real kilobyte, as used by every desktop computer operating system, contains two to the power of ten, 2^10, 1024, bytes. A real megabyte contains 2^20, 1,048,576, bytes. A real gigabyte contains 2^30, 1,073,741,824, bytes. A real terabyte contains 2^40, 1,099,511,627,776, bytes.

As you can see, the difference between the powers of ten and the powers of two - the rip-off factor, in other words - gets worse and worse as capacities rise. Once you get to the terabyte level, the factor is very nearly 1.1.

There can be a further loss of capacity from the space taken up by formatting data - the metaphorical painting of the lines on the parking lot. But that varies with the filesystem you use, and the actual raw capacity you get from a drive with sticker capacity X varies, too.

That capacity is never high enough to cancel out the 1000/1024 rip-off factor, but it often is enough to account for the space taken up by formatting. The "320Gb" Western Digital drives in my current computer do indeed format to 298Gb, exactly what you get if you divide 320 by 1.024 three times. That's thanks to an extra 67-odd megabytes of space, which cancels out the formatting losses. They're still nowhere near 320 real formatted gigabytes, though.

So even if the new "one terabyte" drives are similarly generous, you can only expect them to format to 909 - maybe 910 - gigabytes 0.91 real terabytes, which is 931 real gigabytes.

So, OK, maybe not technically ninety gigs down the toilet. Maybe only 69, depending on which way you look at it.

Either way, that's a lot of $US5000 18 megabyte Winchesters. And there are still plenty of hard drives on the retail shelves that don't hold as much as this new one will rip you off for.

So, until someone starts selling a "1.1Tb" or larger drive, the true 1Tb barrier for single drives will not be broken.

The mismatch, of course, may be getting worse, but it arguably matters less and less, as the price per megabyte of hard drives continues to fall.

But that doesn't mean that people in the year 2020, or whenever, won't feel fleeced when their new "1Pb" drive only formats to a lousy 888 909 terabytes.

Continuation of a theme

Apropos my previous mentions of this sort of thing, I couldn't resist piling on about this Lifehacker post. It's headlined by a simply excellent example of a user interface exploding into a Wain mandala of peripheral detail.

(My answer: A mere 15 active extensions, including of course the excellent ClumsyFingers. Plus a few zombies that don't work with Firefox 2, none of which I've found myself actually missing.)

Click here to be annoyed

It is, indeed, not at all unreasonable to expect an automated system which knows what you did wrong and how to fix it to fix it for you, rather than to just officiously tell you to try again.

On the other hand, if an automated system that knows how something can be done better, yet also knows that you cannot do it better in your current situation, it should avoid encouraging false hope.

USB dumbness

Others have pointed this piece of genius out before, but I think it bears repeating.

Achtung! Eine Schpywaresuche!

The nice people at Prevx, makers of the software which (you may recall) was the only darn thing that saved me from adware purgatory, have a malware database which you can search by filename. This won't help with crapware that generates genuinely randomly named files, but a lot of crapware doesn't do that.

I found Prevx by doing a plain old Google search for the name of an unwelcome DLL, but you can also search it more directly:

Spyware Files SearchType a filename!

The jotti.org scanner lets you actually upload a file for analysis by multiple antivirus programs, while the Prevx scanner just works by filename. But neither of them cost anything, and the Prevx database is better than that of any combination of "antivirus" programs I've seen.

Full disclosure: Prevx don't have an affiliate program, but they've said they'll tip me a buck or two if I send them lots of traffic (and, thereby, generate some sales of the full Prevx1 application. Prevx1 has a fully functional trial period, though, so it can get you out of your current jam for free).

They've also given me a Prevx1 license for free.

I'll try not to spend that all in one place.

NOTE: Prevx have now changed their software so that, like various other commercial spyware killers, it finds infections and then refuses to fix them until you pay up. (And my old license key doesn't even work any more.)

3.8 trees per day

Because I Just Don't Learn about responding to people's news-announcement e-mails, please find herein my review of Uniblue System's "newsworthy and highly topical" LocalCooling software.

LocalCooling interface

(Mmm, alpha transparency. Picture of woogle not included in standard install.)

LocalCooling is software for Windows PC power saving. According to the press release, "if enough people join in, the Local Cooling Community could seriously cut CO2 emissions from computers".

Not that computers themselves actually emit carbon dioxide, of course, but the power stations that run them usually do.

Global warming, LocalCooling. Geddit?

LocalCooling is from the makers of the harmless but largely useless SpeedUpMyPC, which is one of those apps with "273 Five Star Reviews!" because it's got an affiliate program and all the reviews are from penny-ante sites (like mine) that want to cash in (a temptation I've managed to resist. Feel free to throw me a tip in appreciation of my shining incorruptibility).

If you want software to clean your Windows registry, disable useless things in system startup and so on, there's plenty of excellent freeware you can use. Commercial system-speedup utilities often do more than the freeware does, but they very seldom do anything extra that actually needs to be done. Voodoo memory management twiddlers, for instance, are extremely popular among grunty dads and other Power Users, but (a) Windows' memory management has actually been very good in every version since Win2000 and (b) memory's so cheap these days that you'd do better to put the $US30 you're considering spending on memory-optimisation software towards the price of another whole gigabyte of RAM, which'll set you back less than $US100.

Aaanyway, LocalCooling restricts itself to power management. It's got a slick and easy to use interface, but the single functional difference between LocalCooling and the normal Windows Power Options is that LocalCooling lets you set your PC to shut down if it isn't used for X hours (or even minutes, which could be a good way to annoy someone).

To make that setting more useful, you can tell LocalCooling to not shut down before a particular time (presumably resetting at midnight or something; I was never clear on when you were allowed to start feeding mogwai again, either). You can also tell LocalCooling to never shut the computer down if some particular single application is still running.

If you're in the habit of forgetting that you've left your computer on, then I suppose this could be of some use. But it's not as if the standard Windows power saving features don't already have standby/hibernation options. A stood-by PC only draws a few watts, and a hibernated PC draws no more power than it would if it were "off".

Like many modern appliances, PCs are never really "off" while they're still connected to a live power socket. The "vampire" draw of all those little red standby lights all over your house, not to mention the watts used by old heavy linear plugpack power supplies and the smaller but still non-zero number of watts drawn by newer lightweight switchmode plugpacks, can add up to a significant amount of juice.

But I don't have a problem with "off" PCs keeping that one little LED lit on the motherboard and waiting for things like wake-on-LAN signals that will never come, because that extra power also maintains the system BIOS configuration data. Loss of the "CMOS" config is not nearly as big a deal today as it was back in days of yore when losing it meant your computer forgot about all of its hard drives, but it can still be a pain. Modern PCs also have easily replaced lithium coin cell battery backup for CMOS data, instead of the nasty rechargeable or soldered-on lithium batteries that a lot of old PCs had, but that battery will last effectively forever {like, ten years plus} if the PC's got wall power all the time.

Anyway, only if your PC is unable to hibernate is turning it off really a better option. There's no power drain difference at all.

And... that's about it for LocalCooling's actual functions. It lets you set monitor and hard disk sleep timers as well, but so does the standard Power Options.

LocalCooling also has a neat-o tracker that shows you how many "Trees", "Gallons" (of oil) and "kilowatt-hours" you've saved by using it, but this tracker suffers the most common failing of bad science - arbitrary measurements. Not entirely arbitrary, since it's calibrated in watts, but it's got arbitrary fudge factors for the number of watts your PC consumes.

(And also for how those watts convert to trees and gallons, but that doesn't matter much.)

LocalCooling identifies your CPU, monitor, hard drive(s) and graphics card, but it can only actually have a clue about the real power consumption of the CPU and graphics card. Even then it is basically guessing, because it can't tell how hard either is working.

It evaluates my Manchester-core Athlon 64 X2 as being good for 59 watts of power consumption, which is a reasonable enough average figure for a well-used computer. But there's a big difference between idle and working-hard power figures; an easy 50 watts for something like my computer.

Since, like a lot of nerds, I run distributed computing software 24/7, my CPU usage is pegged at 100% all the time and that extra fifty watts ought to be added on.

This is something that a future version of the LocalCooling software could do (this one is only v1.03), since CPU usage is easy to measure and map with moderate accuracy against processor type. But it doesn't do it yet.

The same applies to the video card. Modern high powered 3D cards really are high powered when they're doing complicated stuff in 3D mode, but not when I'm sitting here typing text.

LocalCooling estimates only 20 watts for my GeForce 7800 GT. That actually seems to be something like its idle power consumption, with full-blast 3D mode consumption up around 60 watts (Nvidia specify the 7800 GT as an 85 watt card).

LocalCooling could build a proper database of monitor power consumption, too. They have to depend on Windows' opinion of what monitors are connected being correct, but beyond that they currently just take a wild-ass guess about how much power each monitor consumes. For my 21 inch Samsung CRT they reckon 70 watts, which is OK for a 21 inch LCD, but is a significant under-count for my monitor (which is specified at 125W, but probably more like 100W most of the time). 70 watts would be way off if the "21-22 inch Screen" they detected was actually some old three-foot-deep NEC behemoth.

Oh, and LocalCooling doesn't yet seem to know if you've got multiple monitors. It only notices the primary. That's bad.

LocalCooling makes a couple of less objectionable guesses, too. It assumes each of your hard drives draws eight watts, which may or may not be an overestimate but doesn't matter a whole lot, since the difference won't add up to much unless you've got a bunch of 10,000RPM drives or something.

LocalCooling also tacks on an extra 15 watts for "everything else" - system fans, motherboard chipset power, expansion cards and so on. That's fine; few enough users have a wind tunnel PC full of 30 watt Delta fans.

LocalCooling also doesn't seem to have any integration with the low power modes that various modern processors have. They can step down to a lower clock speed or even voltage on demand. Of course, you should be able to make that stuff happen anyway, possibly with integrated Windows drivers and maybe with extra software from Intel or AMD, but LocalCooling is happy to duplicate the basic Power Options features, so it ought to do this too.

One day we may be running PCs with hardware monitoring that can actually track the current being drawn from the power supply, perhaps even with some granularity so you can see how much juice each subsystem requires.

We're not doing that now, though, so it's impossible for software to really accurately track system power.

LocalCooling could definitely do a considerably better job of it than it does, though. I hope it does in future versions. At the moment all it does is let you join a "community" of people all vying to see how much power they can save. Which is good. But the actual power saving numbers are pretty much random, and LocalCooling does not, currently, actually do anything of importance that you can't do with plain unvarnished Windows.

My Adware Adventure

You know what I did for, oh, about sixteen straight hours, a few days ago?

I hunted adware.

I'm so ashamed.

I ran one little program I shouldn't have. Firefox 2.0 did actually give me its "dude, I really wouldn't download stuff from here if I were you" warning about the site, but I did it anyway. I trusted the file to be harmless just because a couple of virus checkers said so. In the adventure that followed I found out about an "online malware scan" page that lets you submit any file for easy multi-checker analysis - not that that would necessarily have helped.

Anyway, that's all I did. Executed one little program, saw one brief flicker of a command prompt window, started my descent into heck.

Because one little slip like that is, of course, enough to allow the corpsefelching murderbait who make their money by frightening grandmothers into paying for things like System Doctor and WinAntiVirusPro to leap upon my computer in much the same quivering, sweaty way that I imagine they leap upon small children. And, needless to say, their own mothers.

All I got were adware pop-ups and a few dumb-ass toolbars and such desperately attempting to install themselves, but this nuisance-level problem was extraordinarily persistent.

I'm sure some of you are familiar with the symptoms. You run one or another spyware killer, and it finds various problems and gets rid of them (the mania of anti-spyware programs for describing 90% of all known cookies as a screaming-klaxon "infection" is a subject for another day...), but you know you haven't actually dealt with the problem, because weird-named DLLs and EXEs that you can't delete keep popping up in windows\system32. And crap in the registry matching those files' names, of course. You can delete the registry entries, but they always come back, as do the files, if you or your spyware-killing software manage to delete them.

I have, however, finally gotten rid of the problem, by using an excellent tool that I didn't previously know existed. This is probably the outside scoop for most of you, since my skillz are sufficiently 1337 that I haven't even had to think about installing any sort of anti-malware app since Ad-Aware was the one and only option (digression: Word, Ray!). But perhaps you haven't been keeping up with the malware/anti-malware arms race for the last couple of years either, and I've suffered The Curse of 1001 Reboots for a couple of days. So I figure you all ought to share some of the pain.

What I tried before I found the one tool that worked:

Spybot-S&D, which successfully spotted all of the crap being dropped, but did not spot the dropper, so it all kept coming back.

Ad-Aware, which doesn't seem to be much use any more.

Windows Defender, which was worthless. Windows Defender used, of course, to be GIANT AntiSpyware before Microsoft took it over, and people spoke well of that, so I can believe that it's useful for something. Didn't do dick for me, though.

The Ewido online scanner. Which found something, I think, but didn't fix the problem. I have no clear memories of it, since I was hitting my head on the desk pretty hard around then.

Oh, and the Trend Micro online spyware scan, which I gave up on after it had been running for twenty-six hours without finishing.

Avast and AVG, neither of which noticed anything. They're antivirus programs rather than spyware/adware spotters, but these categories are blurring together.

HijackThis, over whose logs I diligently pored. I knew what every single thing in there was, and not one process had anything to do with the churd-gobbling malware.

A Knoppix boot disc, which didn't help much because it can't write to NTFS disks.

A BartPE boot disc, which was more useful, but still didn't really get me anywhere. You can install anti-malware programs as plugins for BartPE, but they generally don't work very well, because they look for malware on the running system. That, of course, is the clean BartPE environment from which you just booted, rather than the dirty hard disk Windows install from which you just didn't.

If you're dedicated enough to put together a BartPE disc containing a registry editor that can load a registry other than the one it booted with, then you can boot BartPE and load the registry from your hard drive and screw around with it. But this was starting to seem like entirely unnecessary effort to me, because I was going to find the people responsible for the spyware and do something to them with, oh, I don't know, maybe a salami slicer, after which I would presumably be put somewhere where my computer would not be available anyway.

Booting BartPE or some other NTFS-capable alternative OS allows you to look at the files created by the malware when they're not multi-locked by important Windows processes (you can only unlock such files by killing those important Windows processes, and then your computer's broken and can't go on to actually do whatever it was that you wanted to do to the now-unlocked files). Looking is about all you can do, though; if you delete them they'll come back when you restart, and many of them are automatically deleted by the spyware as the system shuts down, anyway.

Various spyware uses this horrible strategy now. It's like a highly evolved version of the old Robin Hood And Friar Tuck story.

Oh, and in case you're wondering, yes, I booted into Safe Mode. Oh, boy, did I boot into Safe Mode.

I became quite intemperately angry about all this. My computer is, to a large extent, where I live. Many crapware victims are fairly mystified by even a perfectly working computer and so aren't necessarily especially irked when windows advertising fraudulent antivirus programs keep popping up, because hey, that's just one more thing they don't understand.

When you do understand and expect the correct behaviour of your computer, though, this sort of thing is like someone breaking into your house just to piss on your bed.

And this crapware may be as persistent as herpes, but apart from that it's not even well-written. One of the pop-ups I kept getting was a series of Firefox tabs (which probably wanted to be Internet Explorer windows) that were obviously getting their "URLs" from some file that wasn't being parsed properly. The result was an attempt to open this, and some other HTML header stuff that Firefox I'm Feeling Luckied into http://www.xhtml.com/en/xhtml/reference/, http://www.strict.com/ and http://www.5,.com/.

This made it feel as if the person who kept breaking into the house and pissing on the bed was doing that because he actually wanted to steal the TV, but did not know what a television looked like.

I suppose if you investigate spyware for a living you build up some tolerance for the sheer subhuman exterminability of the people responsible. But I'm not quite there yet. You strap 'em into Old Sparky, I'll throw the switch. Or, more realistically, join the queue for my chance to do so.

Anyhoo, after all this, I stumbled upon Prevx1, when I searched for the name of one of the numerous strange DLLs that kept appearing in my system32 directory.

(Malware writers don't yet, at least, seem to have figured out how to give their files misleading dates. So if you order files by Date Modified, you can easily see the ones that were created on the day when your computer got the clap.)

Prevx1 is a commercial product, but it's got a fully functional trial period - it's not one of those stingy programs that scans for ages, finds a long list of scary problems, then tells you you've got to pay if you want them fixed.

[UPDATE: At some point after I originally wrote this post, Prevx morphed their software into "Prevx CSI", which is now the same "ransomware" as many other commercial spyware killers. It finds infections, but won't kill almost any of them until you pay for a license. I have no idea whether the new version of Prevx currently works any better than the genuinely free anti-spyware options like Ad-Aware and Spybot S&D. Actually, I suspect SUPERAntiSpyware to be the best of the freeware crop, as of mid-to-late 2008.]

It brings to malware-hunting the collaborative user-network approach that's already been employed in spam-fighting. This approach only works better than the traditional kind of virus-definition-file system if you've got a well-connected network of users, but Prevx1 does.

And Prevx1, finally, worked.

It cleared that adware right up, leaving one still-mildly-locked but easily deleted file, and a few deactivated files and pointless registry entries, plus their symptoms like an unconnected Add/Remove Programs entry for some toolbar or other. Oh, and a few more of those cookies that Spybot and the rest think are such a big deal. CCleaner tidied most of the unconnected registry garbage for me.

Anyway, if I'd tried Prevx1 first, none of the other crap would have been necessary. A regular user would be happy with the unadorned result of the Prevx1 scan.

Without Prevx1, though, it would have been damn close to impossible to clean the computer from this one, single, 28-kilobyte-file-induced infestation, without formatting the boot drive and reinstalling.

Since Prevx1 managed to fix it, I presume someone with spare time, an outboard registry editor and a few Sysinternals tools could have done the same thing. That rules out most of the people who're paid to clean up spyware for others, though, and sure as hell rules out nearly every plain old user who would like to clean their own computer.

Plenty of spy/ad/whateverware infestations are less horrible than mine, but I'm willing to believe that a lot of them are a great deal worse, given the enthusiasm of ordinary users for (a) sticking with the default Windows root access and (b) installing every darn thing they see, just to see whether the little Desktop Stripper will get it on with BonziBuddy and the Crazy Frog.

In the olden days, support people who just told callers to reinstall Windows were taking the easy way out. They may have had to do it, given the number of callers they had to get through, but reinstalling was still not by a long shot the optimal recovery strategy for almost any problem.

These days, though, I think it's quite likely that many spyware infestations just can't be fixed by any means less annoying than nuking from orbit. Prevx1 fixed mine, and perhaps it'll go from strength to strength and become the go-to guy for all such problems for the foreseeable future, but I wouldn't bet on it.

Given this fact, and also given the vast amount of time wasted and pain caused by crapware of all kinds, I suppose it would still be uncharitable of me to suggest that the persons responsible could benefit greatly from, say, having a glass turkey baster jammed up their penis, which could then be struck smartly with a club hammer.

I've had a while to develop some perspective now, though, and I'm afraid I really can't see another way.

UPDATE: As I mention here, Prevx have a malware database which you can search by filename.

Herewith, a thingy to do that from here:

Spyware Files SearchType a filename!

File lock detective work

There are eleventy billion little Windows shell extensions out there, but one that I find myself using at least a few times a week is the very directly named WhoLockMe.

When you can't delete (or move, or modify, but usually delete) a file in Windows, it's usually because some application has a lock on it. That's good, because it stops people from pulling the rug out from under running programs, but it's bad, because many processes maintain locks on files that they don't actually need any more.

Windows is not helpful about this problem - all it does is say the file or folder "is being used by another person or program", and suggest you close "any programs that might be using the file."

WhoLockMe lets you instantly see what processes have a grip on any file or directory. And yes, it's smart enough that you can ask who's locked a folder and it'll tell you, even though technically it's a file inside that folder that's locked.

You can even kill the listed processes, right from the WhoLockMe window.

Often, of course, WhoLockMe tells you that it's Explorer that has a lock (or two locks, or eighteen locks) on the file. That situation is unlikely to be resolvable without rebooting, or killing explorer.exe. Windows automatically restarts Explorer when you do that (you can make your computer much angrier than that if you try), but you lose all of your open folder windows and various System Tray stuff.

But it's still nice to know.

Bugger bugger bugger bugger... oh!

My main (WinXP) PC has, of late, developed the habit of hanging when I change screen resolution a couple of times. I get a black screen, or a frozen copy of the last desktop state. The computer's still accepting input, and the mouse cursor knows what it's over and changes appropriately, but there's just no repainting of the screen itself at all. Ctrl-Alt-Del works just fine, insofar as I can tell that there's a Task Manager window in there somewhere by dowsing for it with the mouse, but it's not very useful to me if I can't see what's listed there.

I can, at least, shut the computer down elegantly with a simple press of the power button. At some point during the shutdown the desktop unfreezes - perhaps because explorer.exe is what's hung and it's just exited, but there are still some application windows open for me to look at. Or perhaps not. In any case, it is of course now too late. Huzzah.

Because the screen changes resolution when I play a game, this situation is unacceptable. I can play any game I like - I just have to reboot afterward.

So I reinstalled XP, intending to install over the top of the previous installation and not trample all of my previous stuff.

As any fule kno, you don't do an over-the-top reinstall by selecting the Repair option when the XP installer presents it. That'll give you the Recovery Console, which you probably don't want. Instead, you go on to the next screen and Windows detects the previous installation and lets you either nuke it, install alongside it, or attempt to repair it.

So that's what I did. Except the installer never said it had detected the previous installation. It just chugged on cheerfully installing to C: as if it was a nude blank drive.

Lo, did I say many uncharitable things about this, as I read a book while the chirpy WinXP setup advertisement-screens went past, and tried not to think about all of the stuff I'd have to reinstall and reconfigure (I guess this'd be a chance to switch from Azureus to Utorrent or something, since it certainly can't be any more annoying than recreating a previous list of downloading and downloaded torrents in a new Azureus install...).

And then the system rebooted, and... my previous Windows install came back up again, A-OK. It had done a repair install anyway.

But I still can't change resolution without freezing the screen.

I'll accept that, with abject gratitude, given what I thought I was going to be doing for the next four hours.

But if any of you have any idea why the hell this is happening, do please tell me. I'm flummoxed.

I've tried going to HKEY_CURRENT_CONFIG, System, CurrentControlSet, Control, Video in Regedit and deleting everything therein, and that improved the situation somewhat after I rebooted; now I can at least change resolution and refresh rate freely in Display Settings without the screen freezing. The freeze still happens when I exit any fullscreen app that uses a different resolution, though.

I've also got three pesky Default Monitors (in addition to my two real monitors) that just won't go away. It's like Win98 all over again. I've disabled them in Device Manager, which seems to be the best I can do.

(Yes, I have of course installed the latest drivers for my GeForce 7800 GT. And the latest DirectX, too. Perhaps installing that is what caused the problem. Who knows. I have used all of my search-string-crafting ability and failed to find anybody else who's ever suffered this problem, so I'm just guessing.)

I suppose I should just start disabling startup apps one at a time, despite the obvious irrelevance of almost all of them to the screen display and the fact that all of them Worked Before, And I Didn't Change Anything.

Any better ideas, anyone?

UPDATE: W00t, it's fixed. Hosing out whatever had become of the old drivers with Driver Cleaner Pro, then rebooting and installing the new-as-of-five-days-ago v93.71 Nvidia drivers, seems to have rectified the problem.

The worst that happened was the automatic multi-monitor setup reached the perfectly obvious conclusion that my 21 inch monitor should be running at 800 by 600, while my 17 incher should be at 1280 by 1024, at I think about 12Hz. This was not a major obstacle.

Thanks, everyone!