jducoeur

Following up on my post from last week about mysterious accounts with clearly auto-generated profiles trying to friend me:

The Dreamwidth Anti-spam team (thanks to watersword for pointing me in the right direction), in their reply to me, indicated that they think it's SEO -- Search Engine Optimization -- spam. That makes sense, and I'm kicking myself for not thinking of it.

SEO is all about gaming Google and other search engines, by creating interconnected networks of web pages that look real, which point to some webpage that you're trying to promote. Modern search engines rank pages based on, essentially, a web of trust -- the more pages that, directly or indirectly, point to this one, the more important this one is considered. So SEO companies (and it's a fairly big business) try to build up networks of pages, and get them linked from lots of real pages, in order to raise the ranking of their paid clients. They mostly do this by injecting bogus links anywhere that they can manage it.

These fake DW accounts don't have any content yet, but they're probably playing a long game, building up cross-links. And just the act of following me creates a useful cross-link for them, because it means that my very-real profile has their fake account listed under my "Other Subscribers" section.

So that's a plausible answer to my question of, "What do they get from this?" They are building up plausible-looking networks via those Other Subscribers linkbacks -- even though no sensible person would friend them back, they don't actually care, since they are getting a link anyway. Presumably, once the network is well-established, the 'bot accounts will begin posting spam articles that link to the webpages that are paying them. Google will see that my clearly-real page is linking to my profile, which links to the 'bot, which links to the external pages, and will tend to score those external pages higher since they are part of a real-looking network. That's just a guess (Google's actual algorithms are a well-kept secret, and there is an entire industry devoted to sussing it and gaming it), but it seems reasonably likely.

The moral of the story is, when somebody friends you on DW, always look at their profile -- if it looks bizarre and computer-generated, report it to the anti-spam team. Vigilance on the part of the community is the only way to keep this game from paying off, and if we let them profit from it, they'll start to play this game in more force. We want to nip this stupidity in the bud, and the 'bots are counting on nobody actually trying to deal with it. (This is why they don't want to have any obvious spam content when they friend you: they want you to just leave it alone until it's too late.)

---

Thinking about it a little more, a tiny DW enhancement is probably in order. This problem is precisely why external links from Querki are automatically marked nofollow, and you can't turn that off -- that is the signal to search engines to disregard this link in terms of figuring out a page's importance. This makes Querki useless for most SEO, even the relatively benign sorts; I made the conscious decision that that's not a use case we support.

Any online site (especially any free site) that allows end users to contribute data should use nofollow where possible: otherwise the spammers will find you and overrun you. I learned this the hard way many years ago, when The Rolls Ethereal -- the SCA's online phone book, which I ran in the early days of the Internet -- was destroyed by web spammers.

I believe that DW ought to mark "Other Subscribers" links as nofollow, in order to break the network linkages that make this scam particularly useful. They'd still be able to create SEO accounts, but without those backlinks from clearly-real accounts, Google is probably significantly less likely to score them well. It wouldn't entirely break that game, but it would reduce the economic incentive, without impacting DW in any serious way. In particular, it is correct to mark these links as nofollow, because I can't control those backlinks -- they are forced upon me, so my account shouldn't be considered as linking to them.

---

Oh, and your bonus for reading to the end: today's bit of spambot improvisation, which friended me last night:

Prior to my current job I was analyzing shaving cream in Nigeria. Spent college summers lecturing about mantra to get husband back after separation in Jacksonville, FL. Spent college summers supervising the production of pond scum in Salisbury, MD. Spent 2002-2010 analyzing velcro in Phoenix, AZ. Spent 2002-2009 selling pond scum for the government. Managed a small team researching childrens books in Naples, FL.

There is something singularly appropriate about these SEO scammers supporting the pond scum industry...

Okay -- it isn't by any means perfect, but so far it's the best solution I've come up with.

The only thing I miss, in moving from LiveJournal to Dreamwidth, is the native support for cross-posting from here to FB. So for the past couple of months I've been exploring alternatives. The one I've been using was dlvr.it, as described here -- that's adequate, and makes it fairly easy to post links on FB that point to your posts here.

But the thing is, I don't love that, because not many people actually click through those links. And while I may not love FB, I do have a lot more friends there than on Dreamwidth, so I'd like to be able to actually cross-post, not just link.

For a while, I had thought that the answer was Zapier, and I put in a lot of work getting a true cross-post solution working there. But Zapier has one critical flaw: the approach I'm using for cross-posting requires a feature that only exists in their paid version, and Zapier is insanely expensive. (Like, $20/month.) It's just not worth that kind of money. (Yes, I talked to them about it; they brushed me off and refused to even contemplate a more reasonably-priced tier.) So I gave up and went back to dlvr.it.

But -- as of today IFTTT, the grand old man of the "plug-and-play applications" space, officially opened up their Applet program to all comers: you can build your own tools in it, and yes -- like Zapier, it allows you to insert some JavaScript in the middle.

(Why JavaScript? Because your DW feed is in HTML, and if you just post it directly the results look kind of crappy. I want something better.)

So I've spent a little time in the workday cracks today taking the solution I'd built for Zapier and adjusting it for IFTTT. The experience with IFTTT is a bit different from that of Zapier -- a bit less powerful (in particular, their RSS reader doesn't pick up your DW tags, which Zapier did), but with a much better built-in IDE.

I think that's now working adequately -- it's not The One True Solution, but it mostly works. I've published it as a public Applet on IFTTT; feel free to pick it up and use it. You give it the URL of your Dreamwidth RSS feed, and you need to connect Facebook to IFTTT; once you have that, it should, in theory, quietly check your RSS feed every 15 minutes or so, and cross-post new entries to your Facebook wall. It takes each DW post, translates it into something that looks okay on Facebook (basically, it back-translates the HTML to something vaguely like Markdown), and includes the link to the original DW post at the bottom.

Please pass word on to anybody who might care, and tell me about problems. (Hopefully, I can fix any problems -- once I published, IFTTT gave me dire warnings that I could no longer alter my triggers or actions; hopefully I can still edit the critical filter in the middle.)

Since I'm pretty sure some folks care, here are my findings on cross-posting.

Background: my journal is largely public, and I like to have it disseminated to where people want to read it -- Facebook (FB hereafter), Twitter (TW), whatever. (While I think DreamWidth (DW) is the best place for following one's friends, the reality is that most of my friends are only on FB.) LiveJournal (LJ) has had built-in cross-posting to those services for a long time now, and I've been using that; even after I moved to DW, I've been cross-posting from DW to LJ, and thence to FB and TW. But now that I'm thinking of dropping LJ entirely, the question is how I keep the other services in the loop.

After doing some research (and finding that most of the crosspost-to-FB services have gone away in the past two years), I came to the conclusion that the most robust option seems to be dlvr.it. We are not their primary target market: they are really focused on marketing people who want to be able to write something once and then spew it widely, and their Pro plan is oriented to that. But they do have a Free plan, and their service -- take RSS feeds and post them to social networks -- is more or less what we need. I've been using it for a few days, and it seems to work.

To get this up and running:

• Go to the DreamWidth FAQ about RSS feeds, which should show the URLs of your feed.
• Sign up for dlvr.it. I signed up using my Facebook account.
• Once you're signed up, it will take you their "Automate" page. There, you set up an automation with a "Feed" (the URL of your DreamWidth RSS feed) connected to one or more "socials" (Facebook, Twitter, whatever). The Free plan allows you to take up to 5 Feeds as inputs and 3 Socials as outputs.

That's pretty much it, and it seems to work pretty well; I might even upgrade to Pro eventually, if I decide to use this for official Querki stuff.

That said, some caveats:

• Most importantly, this is a third-party service, and they conspicuously indirect all links through themselves. This stuff is really only for public posts anyway, but keep in mind that they are probably doing traffic analysis on who clicks through to your DW page.
• dlvr.it requires slightly more FB permissions than I love. I believe I understand why they require what they do, but basically you have to agree to all of the permissions required by all of their features, even if you aren't using all of those features.
• dlvr.it is a commercial service, and they support themselves with subscriptions. They really want you to be buying their Pro service, which is expensive. ($10/month) They don't seem to be nasty about Free users, but be prepared to see "Try Pro Now -- Free Trial!" buttons on all the screens.
• They provide a good deal of control over how cross-posts show up, and I had been quite encouraged that there was a "Status Update" option -- that is, cross-post the post in its entirety to Facebook. Sadly, though, it looks like links don't survive the process, so I've backed off to simply posting links.
• The Free plan is explicitly a bit slow in its cross-post speed -- they only check for new posts every half hour. (Unsurprisingly, the plans that cost real money are quicker.)

So -- that appears to be a viable approach to cross-posting from here without LiveJournal in the middle. There are likely other possibilities. (It looks to me like IFTTT is possible, although I'm not sure how well that would work.)

Anybody have other suggestions?

My lady asked me to track down some feeds she'd been reading on LJ.  I found some and wound up creating a few of them, and suspect that other friends might be interested, particularly in:

The East Kingdom Gazette

XKCD

What If?

Popehat

Note that not all of these have populated yet...