The Use of Search History

Today's big deal seems to be a plethora of stories about new AI techniques to be applied to search. There are some good points there, about notions like using facial recognition to better understand photographs, or applying natural language techniques so that the search engines can understand real language instead of requiring "keywordese".

That said, there's still a lot of room for improvements on pure brute-force techniques, if you accept the notion of tracking previous searches. A simple case is progressive search refinement -- essentially playing Blind Man's Bluff with the search results, where you could say "cooler" and "warmer" as you wander through pages, actively triangulating the more and more relevant-looking pages and downgrading the groupings that seem to have less relevance. Another is remembering my previous searches *and* feeding them into the PageRank algorithms, so that pages I previously found useful would increase the network weight of those and related pages. (It's possible that this latter is already being done, but I have no evidence of it.)

Another subtle but serious improvement would be to make the Google Toolbar smarter about paying attention to how I *use* the search results. A page that I click on would be weighted higher than one I didn't. The *last* one I clicked on would be weighted higher than the rest, on the theory that this is typically the one that provided the right answer. Also, pages that I left open for a real period of time would have their weights increased over ones that I closed quickly. This would essentially build PageRank ratings automatically, working around the fact that most people aren't going to go to the effort of clicking on the "good" and "bad" icons.

All of these ideas would be controversial, to be sure -- the privacy implications are quite real, especially in light of last week's AOL search-history debacle. Still, this is an area where there is at least a genuine economic tradeoff of privacy for utility, so the potential privacy loss has some value. (As opposed to many modern privacy intrusions, which have no value at all to the person intruded upon.) I don't take it for granted that everyone would be repulsed by the potential privacy loss -- heck, I'm not even sure which way I'd come down on it. If a company made a real effort to improve the privacy of the history recording (say, by storing URLs as encoded hashes rather as plaintext, so that my search history could not simply be read out later), I could easily see myself accepting the risk...

The Use of Search History

Today's big deal seems to be a plethora of stories about new AI techniques to be applied to search. There are some good points there, about notions like using facial recognition to better understand photographs, or applying natural language techniques so that the search engines can understand real language instead of requiring "keywordese".

That said, there's still a lot of room for improvements on pure brute-force techniques, if you accept the notion of tracking previous searches. A simple case is progressive search refinement -- essentially playing Blind Man's Bluff with the search results, where you could say "cooler" and "warmer" as you wander through pages, actively triangulating the more and more relevant-looking pages and downgrading the groupings that seem to have less relevance. Another is remembering my previous searches *and* feeding them into the PageRank algorithms, so that pages I previously found useful would increase the network weight of those and related pages. (It's possible that this latter is already being done, but I have no evidence of it.)

Another subtle but serious improvement would be to make the Google Toolbar smarter about paying attention to how I *use* the search results. A page that I click on would be weighted higher than one I didn't. The *last* one I clicked on would be weighted higher than the rest, on the theory that this is typically the one that provided the right answer. Also, pages that I left open for a real period of time would have their weights increased over ones that I closed quickly. This would essentially build PageRank ratings automatically, working around the fact that most people aren't going to go to the effort of clicking on the "good" and "bad" icons.

All of these ideas would be controversial, to be sure -- the privacy implications are quite real, especially in light of last week's AOL search-history debacle. Still, this is an area where there is at least a genuine economic tradeoff of privacy for utility, so the potential privacy loss has some value. (As opposed to many modern privacy intrusions, which have no value at all to the person intruded upon.) I don't take it for granted that everyone would be repulsed by the potential privacy loss -- heck, I'm not even sure which way I'd come down on it. If a company made a real effort to improve the privacy of the history recording (say, by storing URLs as encoded hashes rather as plaintext, so that my search history could not simply be read out later), I could easily see myself accepting the risk...

Bwahahahaha!

That unique form of ennui'ed procrastination known as Writer's Block is sort of like taking a long bath in a vat of boiling acid -- it feels so good when you stop.

For months now, I've been staring at the Girl Genius game and getting more frustrated: lots of good characters there, lots of funny potential bits, but there wasn't a game there. A good LARP (at least, one of mine) needs some sort of *story* being told, not just a bunch of random events, and the pieces just weren't cohering. The ideas were floating around in my head, but without some sort of direction I couldn't really write them down usefully.

And then, today, it came to me. I'd had almost all the jigsaw pieces already, but I got the one apparently-unrelated element that connected them. In the course of less than an hour, the overall plot of the story suddenly pulled together. Finally, after staring at the screen for six months, I now know why all of these characters need to be right here, right now. (Not to mention 2-4 more characters abruptly entering from stage left to add themselves to the story. I'm going to be looking for a small group of players interested in taking a bunch of fun but unexpected characters.)

Vast amounts to do yet, of course -- I haven't even begun to write, and probably won't for a fair while yet. But now I know where the game is going, and it's starting to look like fun, with humor, plot and no small amount of angst suddenly added to the mix.

It feels lovely to towel off the acid, and get back to work.

(BTW -- tpau, when are we doing game signups? I probably want to send out character questionnaires the instant the game fills. I want to cast as early as I can manage, to give people time to costume...)

Bwahahahaha!

That unique form of ennui'ed procrastination known as Writer's Block is sort of like taking a long bath in a vat of boiling acid -- it feels so good when you stop.

For months now, I've been staring at the Girl Genius game and getting more frustrated: lots of good characters there, lots of funny potential bits, but there wasn't a game there. A good LARP (at least, one of mine) needs some sort of *story* being told, not just a bunch of random events, and the pieces just weren't cohering. The ideas were floating around in my head, but without some sort of direction I couldn't really write them down usefully.

And then, today, it came to me. I'd had almost all the jigsaw pieces already, but I got the one apparently-unrelated element that connected them. In the course of less than an hour, the overall plot of the story suddenly pulled together. Finally, after staring at the screen for six months, I now know why all of these characters need to be right here, right now. (Not to mention 2-4 more characters abruptly entering from stage left to add themselves to the story. I'm going to be looking for a small group of players interested in taking a bunch of fun but unexpected characters.)

Vast amounts to do yet, of course -- I haven't even begun to write, and probably won't for a fair while yet. But now I know where the game is going, and it's starting to look like fun, with humor, plot and no small amount of angst suddenly added to the mix.

It feels lovely to towel off the acid, and get back to work.

(BTW -- tpau, when are we doing game signups? I probably want to send out character questionnaires the instant the game fills. I want to cast as early as I can manage, to give people time to costume...)