TerraLuna Contract Net Protocol: Open Markets, Open Organizations

The TerraLuna Contract Net Protocol (TCNP) is an peer-to-peer open market protocol which handles postage, encryption, peer training and evaluation, market membership, contract negotiation, data sharing, event sequencing, and project and problem management. It is implemented as a layered stack, and for underlying data transport can use reliable and unreliable protocols of the interactive or store-and-forward variety, including TCP, UDP, SMTP, XMPP, NNTP, and sneakernet.

One feature that sets TCNP apart from most previous protocols is the combination of micropayments and PGP. Micropayments are integral throughout the TCNP stack, discouraging SPAM and DDoS attacks, enabling equitable markets to form between and within organizations, and offering a level playing field for individual participation. The protocol uses PGP encryption and authentication, and leverages the existing public key infrastructure and PGP trust web.

TCNP peers can be human or other agents, including autonomous applications, hosts or other devices. Potential applications described here include a true domain registration market, host management tool, distributed consulting organization, market-based wireless mesh network, and a market for design changes to TCNP itself.

XXX Seriously, this is something I've been working on far longer than infrastructure administration; starting thinking about it almost 20 years ago. Finally understand it well enough to start writing. It's risky for me to mention it here; I still remember how people tore into Steve Roberts (of BEHEMOTH computerized bicycle fame) when he started talking about computerized boats on his technomads list. Some people just like bicycles... But I'm going to go ahead and place myself at your mercy. If it helps, think of this definition: "Infrastructure is the substrate an organization needs to do its job." ...then ask yourself: "How do you build and pay for a reliable, decentralized infrastructure for a non-hierarchical organization?" Non-hierarchical might mean an open-source community, a consortia, a group of industry trading partners, the relationship between consumers and producers, an international NGO... Or the relationship between peer machines in a managed infrastructure. Reliable and decentralized means no single point of failure, and every member carrying their weight without the need for charity or inequitable arrangements such as flat usage fees or membership dues.

A network protocol optimized for mesh communications enables a market-based organizational structure.

XXX

Hierarchical organizations scale poorly. Mesh organizations show potential for much better scalability, but need technologies which support many-to-many communications.

This paper is a living document; revisions and discussion can be found at Infrastructures.Org, a project of TerraLuna, LLC.

The Progress Spot Index (PSI) is the underlying instrument for several Contract Market derivatives. The PSI is valued on a scale of 0 to 100 inclusive, and represents the "doneness" of a project.

Alice is working on a project, and publishes information about it by any means she chooses. She decides to list her project as a progress spot index [spot] on Trent's Contract Market. She writes a description of the project, including the project name, estimated completion date, and any pertinent milestones, resources, requirements, prerequisites, and so on. We'll call this description 'X'. She signs the description with her private key (S_a(X)), encrypts it with Trent's public key, and sends it to Trent (T):

E_t(S_a(X)) -> T

Trent decrypts and reads the description, creates a new ticker symbol (Y) to represent the progress of Alice's project, adds the symbol to Alice's project description, signs the result, encrypts it with the public key of his subscriber network (N), and sends the new listing to his subscribers as part of the market data stream:

E_n(S_t(Y,S_a(X))) -> N

Each subscriber has a copy of the N private key, allowing them to view the data. Bob (B) is one of Trent's market data subscribers. Bob reads the new listing for Alice's project. He knows something about the problem Alice is trying to solve with her project, and decides to become a market maker for the project. He analyzes all of the information he can find about her project and progress, and decides that she is already somewhere between 30 and 40% done. He creates two limit orders, each 5 contracts in size: a bid at 30 (O(bid,Y,30 x 5)), and an ask at 40 (O(ask,Y,40 x 5)). He signs, encrypts, and sends these orders to Trent:

E_t(S_b(O(bid,Y,30 x 5))), E_t(S_b(O(ask,Y,40 x 5))) -> T

Trent verifies Bob's signature, then strips it off so that Bob can remain anonymous. Trent then signs the orders himself, and encrypts and sends the bid and ask orders out on the market data net:

E_n(S_t(O(bid,Y,30 x 5))), E_n(S_t(O(ask,Y,40 x 5))) -> N

Let's take a look at what this gives us so far: We now have a publicly posted project, and a publicly posted anonymous third-party estimate of how far along that project is. Right now the market looks like this:

Symbol	Bid	Ask	Last
Y	30 x 5	40 x 5	-

Now let's see what we can do with it.

Carol has known Alice a long time; they were college roomates. She knows that Alice usually is overly optimistic when it comes to describing how far along she is on a project. When she sees Alice's project and Bob's bid and ask orders go by on the wire, she takes her own look at the public evidence of Alice's progress. By the time she's done, she's pretty sure that Bob has been influenced by Alice's brimming enthusiasm in press releases. Carol reads between the lines, and decides that Alice is actually only 10 to 20% done. Carol decides to sell short with a vengeance. She sends Trent an ask order at 20 points for 90 contracts:

E_t(S_c(O(ask,Y,20 x 90))) -> T

The first 5 contracts in Carol's block of 90 hit Bob's 30 point bid and take it out. Trent notifies Bob and Carol by sending each of them a fill notice (F), showing each what they just now bought and sold:

E_b(S_t(F(B,buy,Y,30 x 5))) -> B

E_c(S_t(F(C,sell,Y,30 x 5))) -> C

This trade leaves Bob holding 5 long contracts which he bought for 30, and Carol short 5 at the same price. Trent records the trade in Bob and Carol's accounts:

Account	Symbol	Trade	Balance
Bob	-	-	$5000
Bob	Y	30 x 5	$4850, 5 Y
Carol	-	-	$7000
Carol	Y	-30 x 5	$7150, -5 Y

Carol's order is only partially filled, so this leaves an ask of 20 still on the market, in front of Bob's ask of 30 x 5. There is no bid, since Bob's got wiped out:

Symbol	Bid	Ask	Last
Y	-	20 x 85 30 x 5	30 x 5

Bob recovers from his consternation and places a new bid at 15:

Symbol	Bid	Ask	Last
Y	15 x 5	20 x 85 30 x 5	30 x 5

So, what have we just seen? A market is a conversation [ref]; in that conversation, the participants continually form and re-form a consensus. In this example, the consensus so far is that Alice is 15-20% done with her project; this can be read directly from the inside bid and ask.

Alice needs money to help fund her project. She's about to publish good news about her project, but nobody else knows that yet. She decides to buy 100 contracts so she can sell them at a higher price later. Alice places a bid order for 100 contracts at 25 points. This hits Carol's 20 x 85 ask, and fills that immediately, leaving the remainder of Alice's bid left over at 25 x 15, in front of Bob's bid. Bob's original ask is now exposed again. The market and accounts now look like this:

Symbol	Bid	Ask	Last
Y	25 x 15 15 x 5	30 x 5	20 x 85

Account	Symbol	Trade	Balance
Alice	-	-	$3000
Alice	Y	20 x 50	$1300, 100 Y
Bob	-	-	$5000
Bob	Y	30 x 5	$4850, 5 Y
Carol	-	-	$7000
Carol	Y	-30 x 5	$7150, -5 Y
Carol	Y	-20 x 85	$8850, -85 Y

Let's look at what just happened. Before Alice's trade took place, the market consensus was that Alice was 15-20% done with her project. After the trade, the consensus is now 25-30%. The consensus has moved upwards, which is as it should be if the project has made progress.

Note that Alice just now made an insider trade -- trading on information not yet public. In the U.S. stock market, this would be considered bad, if not illegal. But here in the progress spot market, this is encouraged; it's a good way of keeping the spot value accurate.

[ask, bid, request, encrypt, response, send, receive, hash, unlock, settle]

Alice has a file which she thinks others might need. It's a large file, and she doesn't want to tie up her 300 baud modem for days at a time, so she dices up the file into small chunks, and encrypts each chunk using a different randomly generated symmetric key (K). Then she sends Trent an ask order for each chunk. Each ask order contains the asking price (P) and description of the chunk. The chunk description contains an arbitrary string (D) describing the file, the SHA1 sum of the entire file (H), the SHA1 hash of the encrypted chunk (U), the random key K, the byte number of where in the file this chunk starts (I), and the pre-encrypted length in bytes (L). Alice signs the order and encrypts it with Trent's public key:

E_t(S_a(O(ask,P,D,H,U,K,I,L))) -> T

Trent decrypts the order, verifies Alice's signature, strips out and stores the chunk hash and random key, then signs the rest and sends it out as market data:

E_n(S_t(ask,P,D,H,I,L))) -> N

Bob needs a file. He knows what the SHA1 hash of the file should be. He sees Alice's ask orders for the file go by in the market data stream. For each chunk, he assembles a bid order at or above Alice's asking price for the chunk. His bid order contains the bid price, SHA1 hash (H), chunk start byte (I), and length (L). He signs and encrypts the order and sends it to Trent:

E_t(S_b(O(bid,P,H,I,L))) -> T

[trent tells alice to send the chunk to bob]

E_a(S_t(Q(B,P,H,I,L))) -> A

[alice sends the chunk to bob, encrypted with K]

E_b(S_a(R(H,I,L,E_k(chunk)))) -> B

[bob computes U and sends it to Trent]

E_t(S_b(V(H,I,L,U))) -> T

[Trent sends K to bob and registers a fill against both orders]

E_b(S_t(W(H,I,L,K))) -> B

[bob decrypts chunk]

[ do the same thing, only also post the chunk hashes -- for files that can be changed] [let exchange also serve chunks, to cut down latency] [no, just treat file delivery and file acceptance as services, have each party publish contracts saying they will deliver/accept and report progress by estimated completion time]

The shim layer adapts the tcnp stack to underlying transports, providing reliable, unidirectional virtual circuits.

The carrier layer routes messages through the virtual circuits provided by the shim layer, and issues, applies, and cancels postage.

The dialog layer XXX

The market layer provides XXX

The contract layer XXX

The shim layer is an adapter between the carrier layer and an underlying transport protocol. The shim layer provides reliable virtual circuits on top of the underlying protocol. The underlying protocol might be reliable, unreliable, connection or datagram oriented, interactive or store and forward; suitable underlying protocols include TCP, UDP, SMTP, XMPP, NNTP, UUCP, HTTP, FTP, and sneakernet. The shim is a link layer; it is concerned only with maintaining a virtual circuit between the source and destination of the underlying transport protocol, and ignores higher-level multihop arrangements such as contract message routing, third-party agents, or market data forwarding.

A virtual circuit might be unidirectional or bidirectional depending on the properties of the underlying protocol. The properties of an open virtual circuit can be detected using vprop().

vbind, vlisten, vconnect, vaccept, vlist, vprop, vtx, vrx, vclose

The shim is the first line of defense against SPAM and DDoS attacks. It does this by sorting pending inbound transport layer source addresses against a graylist score, and then dropping circuits from addresses with low scores when a maximum circuit count is exceeded. The shim receives the circuit count tunable and graylist scores from the carrier layer.

The carrier layer should call the shim's circuitmax() method at initialization, and whenever host load levels change significantly. The carrier layer should call the shim's score() method to adjust the virtual circuit priority for various senders whenever appropriate.

An additional anit-SPAM measure is provided by a "lazy-read" technique; when possible, the shim defers reading of data from a circuit until the carrier requests the data. This means that, in the case of some dropped circuits, only the source address information is ever read from an uninteresting sender.

When the carrier layer is ready to process inbound data, it will call the shim's rx(count) method. This causes the shim to read the next 'count' bytes of data from the highest-ranked virtual circuit and pass it up to the carrier.

After the carrier processes the message and evaluates its worth, it should call the shim's score() method to adjust the virtual circuit priority for this sender.

Here's an example shim implementation using TCP as the underlying transport: A TCP connection is considered to be a virtual circuit. A TCP shim might block on accept() waiting for incoming connections. When accept() returns with a sockaddr stucture, the shim should lookup the sender's address in the graylist, and sort the new connection into the virtual circuit list. If and when the carrier is ready to read a message from this sender, the carrier will call rx(). The shim will call recv() and then pass the received data up to the carrier as the return data for the rx() call.

To establish a virtual circuit, the carrier calls the shim's tx() method

After the carrier processes the message and evaluates its worth, it should call the shim's score() method to adjust the virtual circuit priority for this sender.

The carrier layer provides prioritized, filtered, authenticated, postage-paid message service over the reliable, bidirectional streams provided by the shim's virtual circuits.

[ support COD payments ]

The carrier is the first line of defense against SPAM and DDoS attacks. It does this by [sorting by score, ...] checking the message header for correct postage, and then verifying the PGP signature of the sender. These operations are separated in time by sorting functions; messages are first sorted by postage/payload size ratio, then later by PGP signature distance. [move PGP to contract or dialog layer] The carrier will drop messages off of the bottom of these lists when load factors are high.

[about stamps] [about postage/payload ratio] [about PGP signature distance] [wire protocol -- message format is (from address, to address, statement, question, possible answers) ] [vocabulary: query, reply, ... ] [service query tx API is (URL-like to address, payment, statement, question, possible answers). Carrier then signs, stamps, and sends the message. ]

mkroute, rmroute, lsroute, mlist, mtx, mrx

[check/cancel postage, sort]

[use bidirectional signature paths (from public keyring) for routing; provide for contracts with keypath servers; suggest that sender send TCNP intro mail to fill gaps in routes; provide opportunity to edit sample mail template, sign, encrypt, and send it with keypress; prefer routes where peers have signed route certs; provide for cert servers; i.e. need to replace current volunteer key server infrastructure with key/path/cert servers, all of which are contract net apps] [add postage,... or cancel postage?] [hey! carriers don't add postage; sender does. maybe this should be moved to dialog layer? carriers check and cancel postage on outbound side, only sort on inbound side] [or do I have inbound and outbound swapped around?]

say, ask, XXX

[check signature, decrypt, parse, sort]

[format question or answer, sign, encrypt]

XXX maybe swap contract and market; dialog under market, contract above

The contract layer

[a contract is a single statement/answer set] [a program is made up of one or more interlinked contracts ] [Contract layer maintains a list of all known contracts, who the counterparties are (fingerprint) and how to contact them (url). This is roughly equivalent to an IP routing table, only it's persistent. Entries roll off of this list based on prioritized value to recipient.] [wire protocol]

rfd, comment, rfp, proposal, rfq, quote, po, manifest, invoice, payment, receipt wcontract, find, rcontract, bid,

XXX

[it's all about keyring management -- a market is a distributed public keyring, participants listed on keyring, signatures show membership and authorizations, banking references... a market is a web of trust, a strong set,... message routing via trust paths, those members with lowest MSD are the backbone] [how to do banking and accounting? could use letters of credit, checks, other signable objects, store them on ring or something like ring... something like gpg's trustdb...] [A project is a market.] [When you join a market, a market maker adds a contract for you; you sign this, agreeing to the rules of the market. ] [The members of a market are those people who are counterparties to any contract in the market.] [Only market makers have the private key for a market?] [A subproject is a member of the parent project's market. In this case, the subproject's owner (market maker) uses the subproject's private key to sign the parent project's join agreement.] [If this is true, then the application API "spreadsheet" is the market "board" itself -- the application opens the market, the protocol API returns indexes of the contracts in the market, the counterparties, etc. ] [This means the purpose of the market layer is to maintain these indexes; these are routing tables. ] [The purpose of the market layer's protocol header is to ID the market, the market makers, and any options. Market frames can contain the entire set of contracts, or any fragment. The smallest fragment size is one contract. The market layer is responsible for defragmenting the market.] [wire protocol] [service API]

XXX

DRAFT $Revision: 0.3 $ DRAFT

TerraLuna Contract Net Protocol:

Open Markets, Open Organizations