Thoughts about Nusenet
Jun. 18th, 2023 10:23 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
jducoeurFor a while now, I've been pondering the idea of, "What would / should Usenet look like if we were to rebuild it today?" As Reddit tries to go full Twitter, that topic is getting a little more timely.
So let's take the question seriously, and kick it off with some initial requirements analysis.
(I'm going to post this on both Mastodon and Dreamwidth; comments solicited on both.)
Personal context: Usenet was basically my introduction to the Internet, back in '87: I was one of the founding members of the Rialto, the SCA newsgroup (rec.org.sca), and pretty much lived on Usenet for about five years.
I've been contemplating this "what would a new Usenet be?" for a fairly long time. (I actually own nusenet.org, specifically to provide a home domain if this ever goes anywhere.)
For those going, "WTF is Usenet?", it was the original distributed forum system. Conversations on hundreds of topics, copied from server to server around the world. The tech was primitive by today's standards, but it was fairly cutting-edge then.
So let's think about requirements from a Usenet lens. What did it do well? (+) What were its problems? (-) What were we not even thinking about then?
+ Usenet was topic oriented, not person oriented. That's an important niche, and surprisingly poorly served nowadays.
+ "Topics" could include communities. Some of my favorite newsgroups were for particular niche communities (like the SCA).
+ The topic namespace was hierarchical; you could easily split rec.humor.funny out of rec.humor.
- The Usenet namespace (the list of groups) was controlled by centralized mechanisms that scaled fairly poorly. This worked for hundreds of topics; it wouldn't work for tens of thousands.
(The community quickly devised a workaround, in the form of unofficial "alt" newsgroups for topics that were too new or controversial. These weren't necessarily distributed as widely, but it generally worked.)
IMO, folks should be able to devise whatever groups they want: it shouldn't be centralized.
+ Other than the namespace, the system was highly distributed. Not only wasn't it centrally controlled, it was architecturally almost impossible to control.
(This didn't seem radical at the time, since the other major system was email. Now, it seems kind of radical.)
+ Conversations were explicitly threaded, and threads could branch as needed. No, this isn't obvious, and there are both pros and cons to it.
+ It was defined by the protocol, not by the specific client: more like email, less like Facebook. (Again, this isn't obvious, especially nowadays.)
+ You could block individual posters. For the time, that was a bit radical.
- I suspect the moderation tools weren't nearly good enough for modern requirements, although they were evolving pretty rapidly.
? I'm not entirely sure what moderation means for this sort of medium. Getting this right is important, and not simple. (This is a big topic.)
? While you could avoid reading the messages from a toxic poster, there was no way to prevent a toxic poster from seeing you.
(This was a concept that just plain didn't exist, and still doesn't exist in many systems. But a lot of folks in the Fediverse care about it, so it's worth mentioning and thinking about.)
- Spam was (and is) a problem. Usenet was where we really learned how much of a problem spam could be.
(Yes, this ties into the moderation problem, but is a different problem than bad behavior or toxicity, and probably needs to be looked at separately.)
Okay, that's an initial list, to start the conversation. What have I missed? Do I have some of the plusses and minuses wrong?
For now, let's focus on requirements rather than architecture -- "what do we want?" rather than "how should it be built?" (Or "does this already exist?") Those can come later.
Thoughts?
(no subject)
Date: 2023-06-19 04:20 am (UTC)I agree with this, but with a significant caveat. There will be groups like alt.WeLoveNazis, and the system will need to think about how to react to that. This is something that the fediverse is currently struggling with, I think.
To generalize, usenet had very poor tools for defense against bad actors (because bad actors were legitimately rare at the time). Nusenet would need to be robust against a wide variety of threat models.
(no subject)
Date: 2023-06-19 04:20 am (UTC)Following from that, I would like to propose as a principle that anonymity be explicitly supported, in such a way that the system *cannot* divulge much information to law enforcement and/or totalitarian governments.
(no subject)
From:(no subject)
From:(no subject)
Date: 2023-06-19 12:26 pm (UTC)Not sure I agree with "struggling" per se. It's ongoing effort, and never perfect, but the Fediverse's architectural answer to that seems to mostly work. It accepts that toxic speech will happen -- there's no way to prevent that, in a distributed environment -- so it instead focuses on what amounts to a treaty-based approach: we'll talk to you so long as you enforce behavior rules that we agree with.
There are certainly gaps, and I'm sure there is room for improvement (especially in figuring out how to moderate at scale), but so far I'm cautiously optimistic about the approach.
(no subject)
From:(no subject)
Date: 2023-06-19 04:23 am (UTC)(no subject)
Date: 2023-06-19 09:18 am (UTC)Over in the non-hypothetical world of lobste.rs, someone had the same idea about membership in a forum very much like hackernews -- the social graph being explicit, and public, and used by the moderators in certain cases to not only prune aberrant leaves but also ask probing questions of their invitor. (Also, moderator decisions/actions all go into a public log.)
(no subject)
From:(no subject)
Date: 2023-06-19 12:29 pm (UTC)Yeah, that's a good point. Most systems of communication have issues with this. (Heaven knows this is an area where Mastodon has been fairly weak.)
(no subject)
Date: 2023-06-19 04:29 am (UTC)* Hiding spoilers
* Tags
* Cuts
* Something along the lines of markdown or other *simple* formatting.
** But dear god, no blink or other animated text.
(no subject)
Date: 2023-06-19 07:00 pm (UTC)Personally, I feel that it's a very complicated question. Certainly photos can be of significant benefit to a lot of conversations, but they also can easily dominate conversations and be a mechanism for abuse. They also eat server space. IMO, the old process of linking to photos worked fairly well, but it's not at all ideal in that it doesn't work well for one-off images and requires people to have a place to host photos.
(no subject)
From:(no subject)
From:moderation
Date: 2023-06-19 09:29 am (UTC)Two forms of moderation exist on usenet. The well-known one is the 'moderated group', where every post passes through to a human gateway before being allowed to post. It's a lot of work, and restricts traffic significantly. It works very well at a high cost and low throughput.
The other one is the anti-spam mechanism, which uses public key infrastructure to sign forged cancel messages which chase the spam around the distribution network. It generally did not work well.
Re: moderation
Date: 2023-06-19 06:54 pm (UTC)My guess is that this filtering was also largely responsible for implementing the "moderated group" form of moderation, in that unsigned messages to moderated groups would also be filtered out. (Ideally, they would be rejected by the server where one tried to post them, but relying on that requires trusting everyone else's servers, which generally wasn't the way things were done.)
Re: moderation
Date: 2023-06-20 03:20 am (UTC)"One of the (exceedingly boring) pillars of oppression is to keep oppressed people busy. AKA "just block the person who upset you."
Like if Nazis spend no time blocking and I spend half my time blocking that means the system is serving the Nazis."
Re: moderation
From:Re: moderation
Date: 2023-06-20 11:18 pm (UTC)Re: moderation
From:Re: moderation
From:(no subject)
Date: 2023-06-19 11:40 am (UTC)(no subject)
Date: 2023-06-19 12:37 pm (UTC)AFAIK, Mastodon does it more on an opt-out basis: servers declare which other servers they are going to refuse traffic from. The admin community has been gradually building some consensus lists of bad actors, and IIRC Gargron has finally knuckled under and accepted that the reference implementation should provide each mechanisms to import such lists in an automated way.
I suspect that Nusenet would want to be opt-in as a general rule -- the scale of the modern Internet is big enough that I wouldn't want to assume that every server accepts every group. But it might work similarly to how Mastodon deals with A/V traffic: stuff only hits your server dynamically, when requested. If servers were smallish (as most Mastodon servers are), then opting in to the groups that are requested by your users and not banned could make sense.
(no subject)
From:(no subject)
Date: 2023-06-19 07:03 pm (UTC)I expect that this was fairly important for ensuring that groups had high-quality communities.
(no subject)
Date: 2023-06-19 02:13 pm (UTC)(no subject)
Date: 2023-06-19 08:43 pm (UTC)I would say there's a fair bit of value in the way the rest of Usenet addressed this, which was to not carry anything other than plaintext content, which at least limits the scope of illegal material although it doesn't eliminate it. Even if a distributed system supports images/video/etc., it would probably help substantially to only replicate the image content across servers, and have clients load the images directly from the server they were posted to.
(no subject)
Date: 2023-06-19 05:16 pm (UTC)I believe that the word “troll” was coined to respond to certain types of bad actors in Usenet.
Moderation tools (for moderated groups) can be important. Client-based killfiles are still a good idea, but group level killfiles for moderator use (or the equivalent) would be a good idea. (Welcome back, “plonk”.)
As already noted, a method to keep certain actors from seeing you would be useful.
Server-local groups (some groups are local). Server-local servers (all groups are local).
Moderation comes in multiple levels: pre-post moderation (posts must be approved by a moderator), and post-post moderation (tools available for removing posts). Moderation of messages, moderation of posters (approval, disapproval, removal). Didn’t moderation of posts in Usenet allow moderators to insert notes in approved messages?
Multi-level threading would be good. Easily visible multi-level threading would be good, but I think that that’s a function of the client software more than a function of the groups themselves (FB does multi-level threading, but the client software often stops displaying which comment is being responded to after the first level or so.)
(no subject)
Date: 2023-06-19 06:03 pm (UTC)Yes. Moderators can edit everything about the message; whether and how they do so is up to them. Moderated groups tended to have extensive charter documents stating what was up to mods' judgement and what they had to do.
Disemvowellment as a half-way between discarding a post and letting it through started on Making Light.
I'm not sure where shadowbanning started. (A shadowbanned poster is the only one who can see their own posts/comments.)
(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
Date: 2023-06-19 06:35 pm (UTC)* It was discoverable. Part of this was that although it was large, the group-space was functionally finite. Part of this was that there was some amount of hierarchy. Part of this was being able to follow interesting people to other groups (although I have no memory of how I did that). The result was that I was able to find several great communities about topics that I was interested in, and they were "the" community for that interest.
* The information-transmission system and the UI client were entirely separate. That led to a proliferation of UI clients that met different people's needs. You already mentioned that, but I think one of the critical pieces is that it allowed a lot of programmers to experiment with what made the system work for them, which made for far more innovation than would have happened with a centralized owner of "the UI".
* Local groups were supported using the same system as global groups.
* Although conversations were threaded, there was no assumption that the reader was able to immediately reference the previous message in the thread, and this evolutionary pressure led to a bottom-posting writing style where one explicitly quoted the text that one was replying to before writing the reply.
* More-generally, the time delay in posts was an evolutionary pressure that led to longer-form writing rather than single-sentence replies. This was also supported by the fact that it was possible to reply at any point in the thread -- there was minimal evolutionary pressure to respond quickly before the conversation moved on.
* Notification-wise, replies to threads were equal to "top-level" posts that created new threads. This avoids the problem that Dreamwidth often has, where if a conversation shows up under a week-old post, nobody who's not in the conversation will see it. More generally, although conversations were tree-threaded, the typical reading style was to just read the new posts from the last day, and Usenet readers made this reading style very easy to do. (This feels very rare among threaded forums.)
* Several of the newsgroups I was in died not because of the Inevitable Death of Usenet, but because of specific people who dominated conversations with their personal ideology. They weren't "toxic" in the obvious sense; they were simply deeply committed to an ideology that was at odds with other posters and also capable of writing large volumes of posts, and the result was that conversations in the group became more and more dominated by debating their ideology. All of this was nominally on-topic, and the "death" of the group wasn't visible in a decline in posts, but thoughtful discussion of any other topic got derailed enough that people stopped writing it.
* Groups typically had clear and relatively-easily-findable charters of what was considered on-topic and off-topic. Also, the lack of easy access to back archives meant that well-curated FAQs were common.
(no subject)
Date: 2023-06-20 02:07 am (UTC)If you haven't read it, I commend this article on "toxic influencers". It's not quite specifically talking about the same effect you are, but I think there's some overlap in terms of the personalities involved.
(no subject)
From:(no subject)
Date: 2023-06-19 07:51 pm (UTC)A particularly instructive example I would point to is https://board.spotlighthobbies.com/. It doesn't look like much, and the software is "some janky old Perl script that's incredibly fragile" (as the page footer now notes) that has some really horrible UI misfeatures, but that community has been there for 25 years and is one of the critical hubs of the hobby that it supports, so it is clearly doing a lot of things very well.
The big pieces that it gets right are obviously community. When Tom Carter started the board, he was a well-known figure in the hobby, and before launch he invited a dozen or two other respected people in the hobby to join and participate. That immediately gave the board credibility, so it attracted other people who had valuable things to contribute to the conversations, and it also meant that there were strong positive influences on the group culture from day one.
Also, at the time Tom was running full-page ads for his business in all the hobby magazines, so he put a blurb about the message board in those, which neatly addressed the "discoverability" problem in a way that simply couldn't be done online in that era (and likely not now either).
Tom actively moderated the board, quickly removing inflammatory or off-topic posts. (And, even though he has retired, the new board owners do similar moderation.) There isn't pre-post moderation, but after-post moderation happens pretty quickly, and I think there are IP filters for blocklisting problematic posters -- not ideal, but effective enough. That, manually-updated keyword filters, and exceptionally rudimentary password-based accounts for posting seem to keep the spam out. When an inflammatory or trolling post gets deleted, everything replying to it also gets deleted, and this plus reprimands means people generally don't engage the trolls.
I think it also matters that the board is attached to a retail company. It drives traffic to the business, which means that maintaining its health is a legitimate part of someone's day job. That's not a necessary condition for someone to be a high-touch moderator over multiple decades or for paying the bills to keep a centralized system running, as the Nielsen Haydens have demonstrated, but surely it helps.
There are a few pieces of the software that are notably useful, IMO. It uses a client-side cookie to record a "last read" time, so when you come back later it will tell you which messages are new so you can find them. The "index" view is compact, so you can scan conversations for interesting things easily -- and it has supported an emergent behavior of putting short replies in subject lines, so for a lot of conversations you don't even need to click through to the message. It supports linking to photos, which was really important for a visual-art medium. And it does an interesting sort of balance where the first page of the "index" view contains only recent posts, but within that set it threads things, so new messages in older threads don't get completely buried. This also provides a recency bias to things that people reply to, without restricting it.
Thinking about that, I think I have another item to add to my list of Usenet features:
* Although conversations are tree-threaded, topic drift is supported. Conversations have subject lines, and while the default is for the subject line to reflect the subject line of the previous message, you can also change it. Client UIs usually make this subject change visible in the index page so it's immediately visible when skimming, and some clients will even use it as a signal to consider the message to be the root of a new thread tree.
* Related to that, although thread trees are usually displayed as trees, they are usually presented to people with a bias towards the recent posts (perhaps only showing the unread messages, in a default view), rather than with an unchanging root. IMO, this promotes conversation expansion rather than contraction, which is valuable for promoting conversation in general.
(no subject)
Date: 2023-06-19 08:01 pm (UTC)The Spotlight Hobbies board shows that while there may be features that are important to a functional system, the necessary feature set is quite small. Also, the fact that this board thrived where no doubt hundreds of others using the same software never went anywhere or died out shows that there are critical human-factor elements involved that have little to do with the software.
More broadly, I think the number of things in these lists that are either emergent usage conventions or are things that client UIs have added on top of the base protocol show that supporting emergent behavior is, if not necessary, at least a very valuable component. I would argue that it's better to provide hooks for behavior to emerge (or at least to evolve) rather than to try to define everything in the software. Less is, in this case, better.
(no subject)
Date: 2023-06-19 10:06 pm (UTC)Restricting who can see you is only possible if you don't allow anonymous access. Otherwise, to see posts from the person you're stalking, just log out and try again. However, requiring people to register just to see content is a huge barrier to adoption. So you can't do that by default; at best, you can give people the ability to make their own posts non-public, but in a system with threaded replies and decentralized distribution, you have to figure out what that even means. On Usenet, for example, you might not have seen that post from someone you don't like, but you'd still see the parts that were quoted in replies.
(no subject)
Date: 2023-06-20 03:29 am (UTC)(no subject)
From:(no subject)
Date: 2023-06-20 03:19 am (UTC)I feel like there are a lot of things that are just -standard- these days that would be included automatically just by current convention. Formatted text, mobile interface, embedded media, a11y.
I feel like Discord has a decent basic identity approach: you have an account, but that account can use different nicknames across your different group forums. You can even have multiple accounts to isolate things further. But true anonymity is hard.
(no subject)
Date: 2023-06-20 12:33 pm (UTC)Largely agreed, although I'm conscious of the fact that folks seem to survive without most of these. For example, search is problematic in Mastodon architecturally, even beside the folks who simply don't want it to exist. I think these are all desirable, but I'm trying not to firmly assume them.
(a11y is a little more important than the rest, IMO, but I think is more about the clients than the protocols.)
I'm very conscious of this topic, given how much effort I put into Querki's three-level identity-management system. (Which is pretty much the state of the art under the hood, even though it's never been fully realized in the UI.)
That said, part of me would prefer for identity to be entirely outside the scope of Nusenet per se -- that it should be a consumer of a well-designed identity architecture, not a provider of one.
But the nickname point is a good one (and similar to Querki's concept of "Person"): it's sometimes helpful for the same public Identity to be customized per-community. (Separately from the ability of a single private User to have multiple public Identities.)
As for anonymity, I think folks have done a good job of hashing through most of the points above. Personally, I don't think anonymous posting should be possible -- the history of the Internet suggests that that's a recipe for disaster, although we might want to carve out a niche for opting in to anonymous posting on the small scale (specific posts for a limited time window) for specialized requirements. Anonymous reading is a very different question, and not at all obvious to me: there are arguments both ways, and it could be a per-topic setting.
It's a complex and important topic, and I'm not sure which bits should be part of Nusenet per se, and which belong more properly in a different part of the system.
(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
From:(no subject)
Date: 2023-06-21 06:08 pm (UTC)* is this person allowed to be a user on my site? -- site owner and delegates (let's just assume that for any role there is a revocable delegation power)
* does this group appear on my site? -- site owner
* is this site allowed to send my site articles? -- site owner
* how discoverable is this group? -- group owner
* who can read/join this group? -- group owner
* who can post on this group? -- group owner
* who can create a new group? -- ??
* what prevents group names from colliding -- ??
* what groups do I read? -- user, picking from those available on this site
* what groups do I not read? -- user
* what users do I not see? -- user
* what topics/keywords/scores in a group do I not see? -- user
Users need easy to use but powerful scoring systems to build killfiles.
Group ownership needs to be divorced from being at a particular site.