Electronic message journaling

Electronic message journaling is the process of retaining information relating to electronic messages. In this context, electronic messages are defined as any type of electronic communication data structure. Historically this was an electronic mail, but it may also include instant messages, audio messages (such as those in VoIP), text messages, facsimile messages, or other user collaboration protocol data structures. Beginning about 2005 electronic messages began to include social media that included user-generated content such as blogs, discussion forums, posts, chats, tweets, podcasting, pins, digital images, video and audio files. Several implementation variations exist, altering when, what, and how information is retained.

Background
Archival of electronic messages has become a concern in modern society as regulations and compliance requirements for businesses have become more prevalent with notable Congressional acts, such as Sarbanes Oxley. Other compliance areas of concern are those dealing with U.S. Securities and Exchange Commission (SEC) 17a-4, NASD 3010, HIPAA, the Data Protection Act, and the Patriot Act. Several large corporations lost significant amounts of money because of their failure to meet these compliance requirements. Morgan Stanley had a $1.45 billion judgment against it and Merrill Lynch was issued a $2.5 million fine because of its inability to reproduce e-mail transmissions. Because of growing concerns of similar repercussions, major corporations are implementing electronic message journaling to meet compliance requirements.

Overview
A communication system recognizes and identifies any new outgoing or incoming message. It then creates a journal message containing information extracted from the new outgoing or incoming message. The journal message is then processed for storage while the new outgoing or incoming message is processed normally. Then, at a time of audit, reviewers may search and analyze stored journal messages. E-mail journaling is typically done at the mail server.

Journal message
The journal message contains, at a minimum, the following information: a copy of the content of the actual message, any related metadata such as time, date, and individuals involved in the communication. More information may be included, such as a physical location of the message originator/recipient(s), a computer identifier of the message originator/recipient(s), or a class/category of message. The journal message should maintain the same transport format as the actual message so that existing communication infrastructure can be utilized. For example, an e-mail journaling message will, itself, be an e-mail message containing the journaling information as either attachments or in the body of the journaling message and may be in the MIME format.

Design variations and considerations

 * For real-time journaling, the journal message is sent for further processing at the same time the actual electronic message is being sent. For periodic journaling, the journal message is stored in a secure, local storage area before being archived at the enterprise level on a periodic basis, typically after business hours.  The processing of journal messages after their creation also varies.  A journal message may be forwarded in real-time directly to an archival and storage system, where any storage system processing may then take over.  Alternatively, a journal message may be forwarded in real-time to a journaling mailbox, and then retrieved from the journaling mailbox with periodic extractions to the archival system, where any archival system processing would then take over.
 * Rule-based selective journaling is also well known and in use. With rule-based selective journaling, electronic messages are journaled only if they pass a specific set of rules created by an administrator, possibly relating to specific senders/recipients, keywords, or subjects of the message.
 * When a journaling message uses the same transport format as normal communications, the same infrastructure can be used to transport the journaling message to a preferred destination. In such cases, journaling messages should contain an identifier indicating they are a journaling message and not a normal communication.  This will prevent journaling loops from occurring when multiple mail servers are in use, as a second mail server might receive the journaling message before it reaches the journaling storage destination.

Differences between journaling and archiving
Journaling refers to capturing information about an electronic message while it is in transit. Which messages and the kind of information that is captured should be defined by a system administrator or compliance agent. The journaled message should be encrypted and users should not have access to their own journaled message store. Archiving, on the other hand, is primarily dedicated for backing up communications or removing them from their original location to an off-site location. Archiving generally does not occur while the electronic message is in transit and users may have access to their own archived messages. The journaling system may, however, be used as a communication interface to the archival system.

Microsoft Exchange
Microsoft released a journaling feature in service packs for Exchange 2000 and 2003, during 2004. Microsoft's journaling feature uses real-time journaling to a journaling mailbox. The journaling feature uses a rule-based selection to determine whether an e-mail should be journaled. If any of the sender or recipients, even recipients of an expanded distribution list, have their journaling setting enabled, then the e-mail is journaled.

Lotus Domino
IBM originally released a journaling feature in Lotus Domino 6, during 2002. IBM's journaling feature allows for copying every email as it is processed by the mail server to either a local database at the mail server or a dedicated remote database. IBM's journaling feature uses a rule-based selection to determine whether an e-mail should or should not be journaled. If properties about the e-mail match pre-defined administrator rule settings the e-mail will be journaled.