OTR is a crypto overlay protocol for instant messaging. Instead of encrypting the connection to an instant messaging service like Gtalk, MSN, Skype or ICQ, OTR encrypts messages send over an arbitrary instant messaging service end-to-end. The message leaves your messaging client encrypted, and is later decrypted by the receivers client. Only the communicating clients are in possession of the keys necessary to decrypt the message, and the instant messaging service cannot read the message in clear.

Plausible deniability

Between other very nice properties, the OTR protocol also offers Plausible Deniability as well as Authenticity. This means, that when Alice and Bob are chatting, Alice and Bob can be sure that the messages they receive have really been send by their chat partners, and have not been altered. On the other hand, both Alice and Bob cannot prove to a third party, that any of their chat partner send a message with a specific content.

There is a trivial attack on these kinds of protocols. Assume that Alice chats with Bob, and Bob always asks Alice, to help him rob a bank. Now Alice would like to prove to a judge, that Bob really asks her to rob a Bank. A trivial way of doing this is handing over all keys of Alice to the Judge, so that the Judge can impersonate Alice and say Hi to Bob. Because Bob thinks, he is talking to Alice, he asks her again to rob a bank.

However, Alice might not be willing to hand over her keys to a Judge. Recently, greg found out, that there is a way how Alice can prove to a Judge, that Bob told her to rob a bank, without handing over her private keys. His approach uses Secure Function Evaluation: The concept of secure function evaluation is known for some time now: Assume that you have a function or an arbitrary computer program, that processes two inputs a and b, and generates an output c. Then two parties can jointly compute that function, each providing one of the inputs. The other party doesn’t learn anything about the inputs, and both parties get the output c.

Breaking plausible deniability

Effectively, Alice and the Judge can now jointly compute Alice sides of the protocol. Alice provides as input her private key, and the Judge provides all other inputs, including the messages send by Bob. The output of the jointly computed function can be either the short term communication keys, which Alice and Bob are using in the conversation, or the decrypted protocol messages send by Bob. In fact, this very generic approach can still be optimized exploiting some properties that are specific to the OTR protocol and the DH key exchange used in OTR.

Countermeasures

I assume that it will be very hard to counter this type of attack, because secure function evaluation is a very generic method, that is not bound to any specific properties of OTR.

However, please keep in mind, that this attack is only possible while Bob is still chatting with Alice. As soon as the communication is over, Alice cannot decide to go evil afterwards. Also, while Alice is able to prove the authenticity of the messages send by Bob to the Judge, the Judge cannot prove the authenticity of these messages to another party like a jury.