Pandora Awakens

WebDragon on 2003-06-06T14:46:32

[this was spurred on by Pudge's journal posting and in the end, I decided to make it a new journal entry rather than a reply, so as to elicit more comment.]

Gentlemen, the time has come.

The time is in fact long since past, but intertia and whatnot.

It's plain, clear, and obvious to me that some people simply cannot be trusted not to abuse an open standard. (i.e one of openness and trust)

Thus we see spammers exploiting open relay servers, and the like.

It seems to me that the problem is no longer the spammers or the companies that hire them.

The problem is and remains the seriously outdated trustful mail transport system we all know as e-mail. It is no longer viable as a medium of trusted communication with individuals and organizations electronically. It is time to scrap it.

It is time to scrap it completely, and replace it with something more secure, with tighter and more stringent standards.

And then simply *IMPLEMENT* it as widespread as possible. And leave behind ANYONE not switching to the new system.

It will not be compatible with existing e-mail software or transport agents. All of these will need to be rewritten. WHY? It's obvious if you think about it.

It is the ONLY way to get some lazy admins to implement security and secured communication between client and server. Leave them in the dust, if they do not. I have been pushing comcast for a *long* time, to have encrypted connection and security between the user and the mailserver, PARTICULARLY due to the party-line nature of comcast cablemodem networking. They "can't be arsed" in the vernacular, and it's a source of extreme irritation.

Something must be done, and it is no longer this race to keep up that we and the worldwide mailservers are slowly LOSING because of innefective standards and enforcement. Plus a zillion homebrewed solutions that don't combat the problem itself but merely enact a holding action against abusers of the problem.

registry, without which you CANNOT send e-mail. once registered, abuse gets you REMOVED. (maybe I'm dreaming, maybe I'm not) without registry how do we know you're a trusted user? "everyone gets one."

Maybe I'm just whistling in the dark here, but THINK about it for a while. CAN we replace the e-mail system entirely with something better? It's obvious from the configuration nightmare that sendmail has been, from the growing spam problem, forged addresses, forged headers, HTML e-mail that hogs bandwidth in what used to be a text-only medium including web-bugs to identify you to spammers, viruses (remember when it was IMPOSSIBLE to get viruses from e-mail and the whole AOL meme virus thing? and I mean IMPOSSIBLE. Not without actually opening an attachment. simply viewing a text e-mail CANNOT give you a virus.) and trojans that send e-mail from people that aren't really the owners/users of the computer, open relays being exploited, that something has to give. Yes there are other MTA's out there, but it's still the same old e-mail with the same old problems.

Maybe I'm opening pandora's box here, but without doing so, where's my hope for a better system than what we have now ?

I'm *tired* of this crap.

This is the sound of Mail Transport Agents *crashing* in a completely non-computer-related fashion. Not with a bug, but with me finally unzipping my lip and screaming, BY GOD I've Had ENOUGH!

"Hallelujah! Where's the Tylenol?" :-)


OK, how do we start?

Louis_Wu on 2003-06-06T15:38:42

A few devil's-advocate questions:
  • Will the new system allow people who don't know each other to contact each other? Can I cold-call pudge in the new system?
  • How does one get banned? What social and technical means accomplish the "kicking off the island"?
  • How do we establish standards for "abuse", and how do we change those standards to get around "social hackers"? (See chromatic's journal about google-hacking by bloggers for an example of the need for a good reaction system.)
  • What do we allow through the system? Can we send files, formatted text, auto-executing code? Do we restrict it to plain text? Is unicode plain text? Where do we draw the line, and how do we move the line later?
  • How does one get added to the new system? Do we have a web of trust? Is anyone allowed in until they break the rules? What's to prevent a spammer from joining repeatedly, similar to creating junk email accounts?
  • How do we prevent corporate hijacking? Can we?
  • Do we provide an interface between email and the new system? A gateway similar to nntp-email, nntp-web, irc-$chat or irc-AIM (as exists at PerlMonks for the chat system there). Do we want to make such a gateway easy to use/create, or difficult to create?

And what name do we use for the new system? Several times I wrote "email" and had to re-phrase the sentence. Maybe "efemale"? Digital mail? "email++"? (Despite my feeble attempts at humor, if this new system does happen, it will need some sort of name rather soon - "the alternate" doesn't flow well. And whatever name is chosen initially will likely be permanent; a 'project codename' will become the true name. See NMS for an example.)

...

BTW, the 2nd paragraph in this extract seems to start in the middle of a thought.

Something must be done, and it is no longer this race to keep up that we and the worldwide mailservers are slowly LOSING because of innefective standards and enforcement. Plus a zillion homebrewed solutions that don't combat the problem itself but merely enact a holding action against abusers of the problem.

registry, without which you CANNOT send e-mail. once registered, abuse gets you REMOVED. (maybe I'm dreaming, maybe I'm not) without registry how do we know you're a trusted user? "everyone gets one."

Maybe I'm just whistling in the dark here, but THINK about it for a while. CAN we replace the e-mail system entirely with something better?

Re:OK, how do we start?

WebDragon on 2003-06-06T16:21:20

great questions. I'll chime in where I can, and leave the other answers to people who may know better than I or have better ideas.

last one first; no, it doesn't start in the middle of a thought. "Registry" is the thought. Maybe I should have bulletted it. :-)

on the questions you raised:

1a yes
1b yes

2a Abuse the system
2b To Be Determined

3 A more efficient and effective system of having each user with a "registered" address. A better PGP ? jury's still out on this. Registry database?

4. good questions. I for one don't think the whole html-formatted e-mail allowing web-bugs, embedded scripts, and the like should be permitted. Thanks to Microsoft for this atrocity, before which all we HAD was plain-text and the ONLY way to get an e-mail virus was running an attachment. None of this hidden executable stuff in file.[pif|jpg|dat].exe

5a register (this is just my idea, see?)
5b with said registration, I'd say quite likely yes.
5c- good questions.

6 by stringently enforcing certain standards and not permitting clients/servers that don't follow the standards to communicate over the new channels. There will BE no embrace and extend. Not if we the people have anything to say about it. Either you're 100% standards-compliant or you break the display/transport/encryption/security and can't communicate effectively. (like web pages that won't display properly due to browsers that don't support the standard properly. This should have been enforced better from the get-go. We need a way TO enforce the standards more effective than the debacle that is current web-browsers that STILL don't fully support a 1997-1999 standard. GOD is that ages ago in computer terms! What's WRONG with us that we accept this crap?)

7 IMHO the old system is too broken. MAYBE if it were treated as 'untrusted'. Do we want to allow that? god only knows. this is all speculation at this point and just me ranting about the state of things. :-)

Food for thought, though, eh?

I've deliberately put off any naming ideas until such time as there appears to be concensus with where my ideals and reality are going on this. All I know is that it's got to be one giant trump card and 99% of the existing crap has to be thrown out as a bad idea. Good for its time, but lacking in modern expression and need.

Great feedback though! Much appreciated. Most of your questions would have to be resolved one way or another.

A Simple Refinement

chromatic on 2003-06-06T16:20:04

While you're at it, perhaps you can take five minutes and create a better English than English? This language can be so ambiguous sometimes. It'd be nice to send e-mail that can't be misunderstood.

Seriously, the answer to any given problem is exceedingly rarely "throw it all away and start over from scratch", especially if you have years and years of history.

Re:A Simple Refinement

WebDragon on 2003-06-06T16:27:26

*chuckle* too true, often enough. There is no substitute for clear concise communication.

The problem we're facing now, though is how to get it there, and how to eliminate abuse of the system. The actual content is up to the individual users, and their varying communication abilities.

I understand where you're coming from, but sometimes the history has to take a beating so we can learn from our mistakes.

All too often I've stripped down a Perl program I've written in order to rewrite it from scratch doing things more efficiently, effectively, and with the better code practices I've learned in the meantime. (I'm not saying I always do this with everything, but I have done it several times)

There has to be a better way. The way we have is barely working the way most people want it to, and is ineffective at keeping out the abusers. Ergo, we learn from our mistakes and start over.

Preferably as FAST AS BLOODY POSSIBLE.

Re:A Simple Refinement

chromatic on 2003-06-06T16:41:00

Come back when your Perl program is some thirty years old and is being used by millions of people billions of times a day. Seriously. You're playing with a paper boats in a Navy drydock with that comparison.

Consider XHTML. It wasn't a complete rewrite of HTML 4. It was a refinement. How many people are using it? Consider SOAP. It was a complete rethinking of RPC over the Internet. How many people immediately dropped CORBA or document-based XML or XML-RPC for SOAP? (And do you want to be associated with those people? :) Consider IPv6.

Yeah, you'll solve the spam problem. You'd achieve the same thing much faster by simply unplugging your network cable, though.

Re:A Simple Refinement

WebDragon on 2003-06-06T17:11:27

When you think of how LONG it's taken browsers to get compliant with HTML4 it's not at all surprising to see the foot-dragging about xhtml. *(sigh)* more's the pity.

However, consider this: that this is not a refinement or a rewrite idea but something completely new and different.

When you look at it from this perspective, it sheds a whole new light on the problem. A completely different solution is required to solve the modern problems that the current solution is incapable of doing more than sandbagging a rising flood.

I'm not proposing a rewrite or a refinement. I'm saying junk the whole thing, leave it in the dust and create something completely new and different.

Considering the problems we're having with the existing system I don't think a better one would be a difficult sell at all.

Personally I don't bloody care whether or not we have to drag them in kicking and screaming. It's gotta GO.

Re:A Simple Refinement

chromatic on 2003-06-06T17:34:01

I'm not proposing a rewrite or a refinement. I'm saying junk the whole thing, leave it in the dust and create something completely new and different.

I would count that as a rewrite. I think you and I are at an impasse now. I agree that the current system is flawed in that it makes it trivial to spam. I disagree that throwing away everything of the current system is the right idea — not only are there plenty of good things you'd lose, but it'll take absolutely forever to get it adopted widely. You'd be throwing away everyone who currently uses e-mail too.

Granted, these are just my opinions. I've been completely wrong many times before. I'd probably have said similar things about Gopher and Archie a mere decade earlier. If you really think this is the right approach, please go ahead. If it works out, I'll use your system and I'll buy you at least two drinks of your choice if we meet up in person. Best of luck.