How to Spot a Psychopath

This Consumerist piece is a good summary of the not-terribly-secret world of the press embargo, which in the computer world manifests itself in those sudden snowstorms of online reviews that show up for each new piece of PC gear, all on the same day and nearly on the same hour.

There are, as the Consumerist piece makes clear, some perfectly valid reasons for embargoes to exist. But they're mainly just another way for the makers of news to control the journalistic process, just as the precious gift of "access" prevents journalists at White House press conferences these days from saying... well, anything much.

The whole embargo/Non-Disclosure Agreement thing has pretty much passed me by, since I'm not a worker bee for a big hardware site, and I always stick to the Never Sign Anything rule - which is also a good policy when confronted with forms that say that if the $20,000 devkit PlayStation 4 you've just been handed breaks while in your custody, you have to pay for it. Frankly, I'm surprised anybody ever signs those.

Anyway, here's my embargo story.

I broke the embargo on the Pentium 4, back in late 2000 when I was still working for the Dark Lord Murdoch.

Well, technically perhaps I didn't break the embargo, because nobody ever made me sign anything; I never even saw an embargo form, though I was of course aware that Intel meant there to be one.

What happened was, a new and exciting P4 computer (except not really, what with the low initial clock speeds and the Curse of Rambus Memory) showed up in our chaotic New Economy office. It was addressed and delivered to someone in the advertising sales department, let's call him Pete, who had no idea what to do with it, though he thought it might fetch a lot of money on eBay.

Pete asked the General Manager what to do. The GM knew I wrote reviews and so sent him to me.

I was pleased with my new toy.

I also observed that there was no Non-Disclosure Agreement form to sign in, on or around the computer's carton, and asked Pete if he'd signed anything.

He hadn't.

So I took the computer home, ripped through the review at warp speed, and published it on the Australian IT site, scant hours before the embargo deadline every other reviewer was sticking to.

(You won't find the review there any more, or anything else I wrote. That's because of the goldfish-like memory of various news.com sites, which I've mentioned before. The review's here on dansdata.com, though, in case you want to relive the days when 256Mb of RAM for a P4 cost one thousand eight hundred Australian dollars.)

As it turned out, there was an embargo notification included with the P4. It was one paragraph in the middle of the cover letter, which I don't think Pete ever gave me. Intel later agreed that a larger font size, and perhaps even investing in a colour printer, might have been an idea, while they revamped their system to make sure they didn't keep sending review hardware to ad men, graphic designers or the muffin delivery guy.

The review, as you'd expect, was massively popular... for the several hours it was up, before Intel Australia told us to take it down again until after the embargo time.

Since it wasn't exactly a Watergate-level story, we did. It would have been pretty funny if we told them to get knotted, left it up, and then sat back to see whether they refused to have any further dealings with the tiny and unimportant News Limited media channels, though.

Intel (or at least their PR people) weren't actually noticeably upset about the whole thing, so it would have been churlish to turn it into a big argument.

Then, though, a moderate amount of hell broke loose, with a few other hardware writers castigating me for not playing the game. One awfully famous fellow insisted that I simply had to have signed an NDA, most likely in blood, and then deliberately broken it, and was therefore a lying son of a bitch and could forget about ever seeing any links from his site to mine ever again. Until he forgot all about it, and all of the other sites he'd blacklisted for various other real or imagined sins, a year or three later.

I also received feedback from one of the hacks at The Register, where actual journalists work.

She said obviously The Reg would have run the story if they'd had the chance, and as early as possible too. What sort of pillock wouldn't?

You can tell a real journalist by the nicotine stains, the cirrhosis of the liver, and the refusal to treat PR people with the respect they don't deserve.

A journo's a pretty low form of life, but there are plenty lower.

I just resurrected one of my old articles from when I worked for the Dark Lord Murdoch. Most of the stuff I wrote back in the Before Time of the dot-com boom I also published on dansdata.com. I think there was some kind of News Limited copyright contract that expressly forbade that; I dealt with that problem by never signing it. But some articles were News Interactive only, and thus disappeared into the ether.

The sporge article was copied with permission here and there, but it vanished quickly from the AustralianIT site because, wait for it, they didn't archive old material.

At all.

It was as if Rupert Murdoch could only afford a two megabyte Geocities page, or something. Stuff just got thrown on the floor after a month or three.

Oh, and the article URLs were encrusted with gunk by the Content Management System. There'd be lots of gloriously typeable stuff like http://www.australianit.com.au/common/storyPage/0,3811,1588489%5E504,00.html.

Those attuned to the pre-blog Force will recognise these URLs as being the unvarnished output of the Vignette Content Manager. Many sites used Vignette back then, because they did not listen to Phil. But those with a bit of civic pride put something between Vignette and the world that turned the URLs into something non-horrible.

We didn't.

And, when I was there, our site had no search function. Not even for the miserable quantity of content that was on it at any given time.

Oh, there was a search that we could use. And the people who sold Vignette to News showed, I believe, a feature list with a great big tick in the "searchable" box, obviously, because what kind of cockamamie CMS makes a site that can't be searched.

But, as it turned out, the only way to actually search the content of a Vignette site at the time was by doing something like signing in each searching user as a user of the actual Vignette server itself. This, for no readily identifiable reason, rapidly paralysed the very expensive servers that were all humming away on the lower floors because Rupert (or more likely Lachlan, who was in charge of the whole Australian online endeavour, but ain't no more) could not abide the concept of off-site hosting.

At the time, I got the impression that Vignette's advice to people who were wondering why their $250,000 server couldn't handle the traffic of their medium-popularity site, which Apache could handle happily on a single processor 400MHz P-II box, was "buy a $500,000 server".

Yeah, yeah, I hear you say. So far, so unremarkable. Messed-up stuff happening during the dot-com era in places where the big bosses made eleventy-three figures and people with piercings played pool and Dreamcast when they were meant to be working. There were only about a million of those stories, right?

The punchline, though, is that the Australian IT site is still, seven years later, now that the Web is more than twice as old as it was then, exactly the same.

Check out this recent piece.

(Or don't, if you're reading this long enough after I posted it, 'cos it won't be there any more!)

Ugly URL? Check.

(Apparently the latest, version 7, edition of Vignette finally kills the comma-ed monster URLs. Either News still isn't running that version, or they're running it in backwards-compatible mode.)

No search box? Check.

And don't expect to be able to find anything by using that new-fangled Google or anything, either, because every page has a big bold <META NAME="ROBOTS" CONTENT="NOARCHIVE"> header, which tells search engine spiders to bugger right off. And there's a matching robots.txt, of course.

All of the other news.com.au pages have the robot-repellent, too. Though they add a trailing slash, 'cos they're cool.

News.com.au itself is, to be fair, not quite as bad as my old sub-site (which was australianit.com.au at the time, but is now a news.com.au sub-domain).

News.com.au has a search box, and an archive that goes back a whole thirty days - a fact I determined by searching for the name of Australia's prime minister, then chugging through to the oldest article, which when I found it was 29 days, 23 hours and 58 minutes old. See if it's still up when you read this - the searcher has, of course, now lost it).

But the searcher only searches news.com.au pages, not the australianit.news.com.au subdomain. Search for computery stuff, find nothing.

News Corporation, of course, owns a buttload of other newspapers and TV stations and such.

The UK Sun's site betrays in its headers that it's still running Vignette version five (with, of course, the commaed URLs to match), and it's got the NOARCHIVE tag too, plus an entertainingly officious robots.txt. It's got its own search box, though, and that seems to give access to miles of old articles.

Rupert owns the UK Times and News of the World, too. The Times is the same deal - NOARCHIVE again, but with an, um, archive of articles. But NotW breaks ranks, with less painful URLs, search engine archiving allowed, but no search box of its own.

(I wonder if someone sweated blood to get that to happen? There's a bloke called Dave whose e-mail address is at the top of the source of every NotW page. Perhaps he'll tell you.)

And then there's the New York Post, another News property. It appears to be entirely untouched by the pernicious spread of Vignettery, and behaves as a newspaper site should; 1337 h4XX0rZ may enjoy probing the short list of directories that robots.txt wants Google to leave alone.

Back here in the land of jumbucks and billabongs, though, News Interactive seems determined to keep itself as irrelevant as possible, by preventing people from finding articles current and old, even if they're willing to pay for the privilege.

This seems, on the face of it, to be a strange decision for Murdoch to make. Whatever else he may be, he's not stupid, and someone who's supposed to be so gosh-darned enthusiastic about bending the world's opinions to his will would, you'd think, be more keen on letting people read what his minions write.

If you excise yourself from the search engines and don't even keep a pay-for-access archive of old stories, then as far as the Web's concerned, you pretty much don't exist.

Then, however, it hit me.

All of the Web-excised newspaper sites may say things that Rupert wants the world to believe, and they may not. Rupert's flagship educator-of-the-proletariat, though, is not any of these mere papers; it is the entirely unbiased Fox News.

The Fox News site has Vignette URLs (and that Version 5 tag, too).

But it also has a big old archive, and no ROBOTS taggery at all, beyond this. Google indexes all of its pages just fine, just as it does with the similarly conservative New York Post.

So, in the end, all this is not WTF-worthy at all. It's part of the plan.

So take heart, everyone who still works for one of the News properties whose Web sites are hidden from the world.

Even if you have a hard time looking at yourself in the mirror these days, Rupert still does not believe you are trustworthy. Even The Sun isn't on-message enough any more.

I bet you didn't feel like a bunch of dissidents, did you?