ESAPI 2.0 Symmetric Encryption User Guide

ESAPI.properties Properties Relevant to Symmetric Encryption

Those properties that are new since ESAPI 2.0-rc2 are shown in red. Values shown in blue are ones that you would replace.

ESAPI.Encryptor

org.owasp.esapi.reference.crypto.JavaEncryptor

Encryptor.MasterKey

Encryptor.MasterSalt

Encryptor.EncryptionAlgorithm

AES

Encryptor.CipherTransformation

AES/CBC/PKCS5Padding

Encryptor.EncryptionKeyLength

128

Encryptor.MinEncryptionKeyLength

128

Encryptor.ChooseIVMethod

random

Encryptor.fixedIV

0x000102030405060708090a0b0c0d0e0f

Encryptor.CipherText.useMAC

true

Encryptor.PreferredJCEProvider

<empty string>

Encryptor.cipher_modes.additional_allowed

CBC

Encryptor.cipher_modes.combined_modes

GCM,CCM,IAPM,EAX,OCB,CWC

Encryptor.PlainText.overwrite

TRUE

Encryptor.KDF.PDF

HmacSHA256

Encryptor.CharacterEncoding

UTF-8

How the Old (Deprecated) Methods Were Used

To encrypt / decrypt using the String-based, deprecated methods carried over from ESAPI 1.4, code similar to the following would be used.

    String myplaintext = "My plaintext";
    try {
        String ciphertext = ESAPI.encryptor().encrypt(myplaintext);
        String decrypted  = ESAPI.encryptor().decrypt(ciphertext);
        assert decrypted.equals(myplaintext);
    } catch(EncryptionException ex) {
        // Log error then return error designation however appropriate
    }

This code will still work, however if you are using the standard (default) reference for ESAPI.Encryptor, which is org.owasp.esapi.reference.crypto.JavaEncryptor, the cipher transformation used with be that specified by the property Encryptor.CipherTransformation with a key size (when the algorithm supports a variable key size) of that specified by Encryptor.EncryptionKeyLength and the IV type specified by Encryptor.ChooseIVMethod. What is not provided by these methods (and why they are deprecated) is that they provide no mechanism to ensure message authenticity unless they are used with a so-called “combined” cipher mode such as CCM or GCM. (Note that as of JDK 1.6, the default JCE provider, “SunJCE”, does not support any combined cipher modes.)

Requirements for using ESAPI Symmetric Encryption

The parties participating in using ESAPI's symmetric encryption capabilities must agree on the following:

1. All parties must share the same encryption key(s). If multiple keys are used, each party must know which key to use for encryption / decryption. Parties must ensure that exchanging, storing, and all management of these encryption keys is done securely. How this is done is presently outside the scope of ESAPI encryptions, but developers may find the advice in OWASP's Cryptographic Storage Cheat Sheet helpful in this regard.
2. If parties are only exchanging raw ciphertext (plus IV, when appropriate), then they must also agree on the cipher transformation to use.
3. If the involved parties participating in encryption are not using a "combined" cipher mode that provides both confidentiality and authenticity (for example, something like GCM or CCM), then they must whether or not there will be an additional MAC sent allow with the raw ciphertext and other cipher spec information in order to provide evidence of authenticity / data integrity. (See the property Encryptor.CipherText.useMAC for further details.) Failure to agree on this may allow an adversary to carry out certain chosen ciphertext attacks against their encrypted data resulting in (possibly complete) leakage of the corresponding plaintext.

Encrypting / Decrypting with the New Methods -- The Simple Usage

Using the new encryption / decryption methods is somewhat more complicated, but this is in part because they are more flexible and that flexibility means that more information needs to be communicated as to the details of the encryption.

A code snippet using the new methods that use the master encryption key would look something like this:

    String myplaintext = "My plaintext";
    try {
        CipherText ciphertext =
            ESAPI.encryptor().encrypt( new PlainText(myplaintext) );
        PlainText recoveredPlaintext = ESAPI.encryptor().decrypt(ciphertext) );
        assert myplaintext.equals( recoveredPlaintext.toString() );
    } catch(EncryptionException ex) {
        // Log error then return error designation however appropriate.
    }

Yes, this is a bit more complicated, but it will 1) work across different hardware platforms and operating systems whereas the older methods may not, and 2) it provides for authenticity and confidentiality of the ciphertext regardless of which cipher mode is chosen.

Also, these new methods allow a general byte array to be encrypted, not just a Java String. If one needed to encrypt a byte array with the old deprecated method, one would first have to use

    byte[] plaintextByteArray = { /* byte array to be encrypted */ };
    String plaintext = new String(plaintextByteArray, "UTF-8");

all the while catching the required UnsupportedEncodingException. For example, to handle this in ESAPI 1.4, one would have to write something like:

    try {
        byte[] plaintextByteArray = { /* byte array to be encrypted */ };
        String myplaintext = new String(plaintextByteArray, "UTF-8");
        String ciphertext = ESAPI.encryptor().encrypt(myplaintext);
        String decrypted  = ESAPI.encryptor().decrypt(ciphertext);
        byte[] recoveredBytes = decrypted.getBytes(“UFT-8”);
        assert java.util.Arrays.equals( plaintextByteArray, recoveredBytes );
    } catch( UnsupportedEncodingException ex) {
        // Should not happen but need to catch and deal with it anyhow.
        // Log error then return error designation however appropriate.
    } catch(EncryptionException ex) {
        // Log error then return error designation however appropriate.
    }

However, dealing with this in ESAPI 2.0 is not any more cumbersome than dealing with Strings:

    try {
        byte[] plaintextByteArray = { /* byte array to be encrypted */ };
        CipherText ciphertext =
            ESAPI.encryptor().encrypt( new PlainText(plaintextByteArray) );
        PlainText recoveredPlaintext = ESAPI.encryptor().decrypt(ciphertext) );

        assert java.util.Arrays.equals( plaintextByteArray,
                                        recoveredPlaintext.asBytes() );
    } catch(EncryptionException ex) {
        // Log error then return error designation however appropriate.
    }

Ideally when you are encrypting sensitive data you do not want the plaintext sensitive data to be left lying around after it is encrypted. Instead, you should overwrite them after their value as been used. However, when you are using immutable Strings, this is not possible using native Java methods. But if you are able to pass in byte arrays that are passed directly to PlainText objects (as shown above), the default is to overwrite this after they are encrypted. (Note: Verify!) If the default for Encryptor.PlainText.overwrite of true had been used, then the array plaintextByteArray would have been overwritten with ASCII “*” characters.

Encrypting / Decrypting with the New Methods – Storing Encrypted Data

If you use one of the new Encryptor encrypt() / decrypt() methods, how do you persist the CipherText object returned by the encrypt() methods and how do you restore it to pass to the decrypt() method?

The following example code snippet will illustrate this. In the following example we will simply write out the serialized CipherText object to a local file, but obviously you could hex- or base64-encode the serialized byte array and store it in a database or sent it in a SOAP XML message to a web service, etc.

    public class PersistedEncryptedData
    {
        public static int persistEncryptedData(PlainText plaintext,
                                                String filename)
            throws EncryptionException, IOException
        {
            File serializedFile = new File(filename);
            serializedFile.delete(); // Delete any old serialized file.

            CipherText ct = ESAPI.encryptor().encrypt(plaintext);
            byte[] serializedCiphertext = ct.asPortableSerializedByteArray();

            FileOutputStream fos = new FileOutputStream(serializedFile);
            fos.write(serializedCiphertext);
            fos.close();
            return serializedCiphertext.length;
        }

        public static PlainText restorePlaintext(String encryptedDataFilename)
            throws EncryptionException, IOException
        {
            File serializedFile = new File(encryptedDataFilename);
            FileInputStream fis = new FileInputStream(serializedFile);
            int avail = fis.available();
            byte[] bytes = new byte[avail];
            fis.read(bytes, 0, avail);

            CipherText restoredCipherText =
                                CipherText.fromPortableSerializedBytes(bytes);
            fis.close();
            PlainText plaintext = ESAPI.encryptor().decrypt(restoredCipherText);
            return plaintext;
        }
    }
  

Advanced Usage

Encrypting / Decrypting with the New Methods

ESAPI 1.4 and earlier only allowed you to use the master key (MasterPassword in ESAPI 1.4; Encryptor.MasterKey in ESAPI 2.0) to encrypt and decrypt with. But encryption with a single key seldom is sufficient. For instance, lets say that your application has a need to encrypt both bank account numbers and credit card numbers. The encrypted bank account numbers are to be sent to one recipient and the encrypted credit card numbers are to be sent to a different recipient. Obviously in such cases, you do not want to share the same key for both recipients.

In ESAPI 1.4 there was not much you can do, but in ESAPI 2.0 and later, there are new encryption / decryption methods that allow you to specify a specific SecretKey. There is also a static helper method in CryptoHelper to allow you to generate a SecretKey of a specific type. (Distribution of this key is out of scope for this particular example, but for the moment, we will assume that secret keys are first generated, and then distributed to the recipients out-of-band. On you could distribute them dynamically via asymmetric encryption assuming that you've previously exchanged public keys with the recipients.)

The following illustrates how these new methods might be used.

First, we would generate some appropriate secret keys and distribute them securely (e.g., perhaps over SSL/TLS) or exchange them earlier out-of-band to the intended recipients. (E.g., one could put them on two separate thumb drives and use a trusted courier to distribute them to the recipients or one could use PGP-mail or S/MIME to securely email them, etc.)

    // Generate two random, 128-bit AES keys to be distributed out-of-band.
    import javax.crypto.SecretKey;
    import org.owasp.esapi.crypto.CryptoHelper;
    import org.owasp.esapi.codecs.Hex;

    public class MySecretKeys {
        public void main(String[] args) {
          try {
            SecretKey bankAcctKey = CryptoHelper.generateSecretKey("AES", 128);
            SecretKey credCardKey = CryptoHelper.generateSecretKey("AES", 128);

            System.out.println("Bank account key: " +
                Hex.encode( bankAcctKey.getEncoding(), true ) );
            System.out.println("Credit card key: " +
                Hex.encode( credCardKey.getEncoding(), true ) );
          } catch(Exception ex) {
            ex.printStackTrace(System.err);
            System.exit(1);
          }
          System.exit(0);
        }
    }

Second, these keys would be printed out and stored somewhere secure by our application, perhaps using something like ESAPI's EncryptedProperties class, where they could later be retrieved and used.

In the following code, we assume that the SecretKey values have already been initialized elsewhere.

    SecretKey bankAcctKey = ...;        // These might be read from EncryptedProperties
    SecretKey credCardKey = ...;        // or from a restricted database, etc.
    ...
    String bankAccountNumber = ...;     // Assume obtained elsewhere
    String creditCardNumber = ...;      // Ditto
    ...
    try {
        // Encrypt each with their appropriate secret key
        CipherText encryptedBankAcct =
            ESAPI.encryptor().encrypt( bankAcctKey, new PlainText(bankAccountNumber) );
        CipherText encryptedCreditCard =
            ESAPI.encryptor().encrypt( credCardKey, new PlainText(creditCardNumber) );
        ...
        // Decrypt using appropriate secret key
        PlainText recoveredBankAcct = ESAPI.encryptor().decrypt( bankAcctKey, encryptedBankAcct ) );
        assert bankAccountNumber.equals( recoveredBankAcct );
        ... etc. ...
    } catch(EncryptionException ex) {
        // Log error then return error designation however appropriate.
    }

Using ESAPI with Multiple Cipher Transformations

For most things, this works well. Most applications likely can standardize on a single cipher transformation such as AES/CBC/PKCS5Padding with a 128-bit AES key and use that 100% of the time. However, on occassion, an application may need to use two separate cipher transformations (or even two different cipher algorithms) to handle legacy applications or deal with multiple partners.

This section discusses how to do this without implementing your own classes or extending the ESAPI reference class, JavaEncryptor. Note that it is recognized that this approach is somewhat of a kludge. A simpler approach is planned for ESAPI 2.1, but the approach shown here is workable even though it's not pretty.

If you find yourself in need of encrypting with a different cipher transformation, the first thing that you should count on is not using the same encryption key for each. While in some cases this likely would work (e.g., you are only using a different cipher mode or you have 256-bit AES key for "Encryptor.MasterKey" but also have a need to do encryption with a 128-bit AES key), it is not guaranteed to do so. Instead, you should count on generating a separate encryption key and using the encrypt / decrypt methods taking an additional SecretKey parameter as show in the previous section.

Rather than repeating all the details of how to do this in this user guide, we encourage you to investigate how this was done in the Junit testing for the JavaEncryptor class. Please look at the source code for the private method runNewEncryptDecryptTestCase(String, int, byte[]) in the EncryptorTest JUnit test in the source code "src/test/java/org/owasp/esapi/reference/crypto/EncryptorTest.java". This code calls:

		ESAPI.securityConfiguration().setCipherTransformation(cipherXform);

There are also a few non-obvious key size adjustments that are also going on with DES and DESede (aka, triple DES) keys that are made there as well. This has to do with the fact that for DES keys (which includes DESede), the "true" key size differs from the "effective" key size. (E.g., in DES, the "true" key size--from the DES algorithms's perspective is 64-bits, however the effective key size for DES is only 56-bits because of the 8-bits of parity "imposed" by the NSA in the early 1970s.) This inconsistency manifests itself in the JCE by the fact that KeyGenerator.init() wants the effective key size to be specified (so 56-bits for DES, 112- or 168-bits for DESede), but SecretKey.getEncoding().length stores the "true" key size (e.g., 64-bits or 192-bits for DES and DESede respectively). Other cipher algorithms do not have this discrepancy between true and effective key sizes. Since a SecretKey is returned by the KeyGenerator, this is only all too confusing. The reference JavaEncryptor and its JUnit test, EncryptorTest attempt to deal with this discrepency. See the adjustments made to key size in EncryptorTest.runNewEncryptDecryptTestCase() for further details.

Using ESAPI Symmetric Encryption with FIPS 140-2 Cryptographic Modules

1. Follow the vendor's regular instructions to configure your application to use the vendor product in a FIPS 140-2 compliant manner. Generally this will include some special configuration and/or initialization requirements that must be followed to restrict the vendor software to use FIPS 140-2 approved algorithms.
   
   If the vendor's instructions do not already include a recommendation to edit the $JAVA_HOME/jre/lib/security/java.security file, the OWASP team recommends that you do so to prevent you from accidentally using any other cryptographic module (such as SunJCE). To do this, you will need to change the property "security.provider.1" (which represents the default) provider so that it refers to the fully-qualified classname of your vendor's JCE Provider class. (By default, this is set to "sun.security.provider.Sun".) In fact, you may wish to go as far as to either comment out or remove the other providers to reduce the possibility that they will accidentally be used by your application.
2. Edit the ESAPI.properties that your application is going to be using and set the property "Encryptor.PreferredJCEProvider" to the fully qualified classname of your vendor's FIPS 140-2 compliant JCE provider class. (If you are using a Java SecurityManager you may also need to grant org.owasp.esapi.reference.crypto.JavaEncryptor permissions to change the JCE provider.)
3. Edit the ESAPI.properties that your application is going to be using and set the property Encryptor.CipherText.useMAC to false. This is critical as having this property set to true causes ESAPI to use derived keys for the actual encryption and MAC calculation, which is against FIPS 140-2 compliance. (Note: This does not mean that OWASP believes that this key derivation is weak--its design has been suggested some cryptographers--but it would be creating and using a related key for encryption in a manner not reviewed by NIST and thus it is not acceptable to FIPS 140-2 approval.)
4. Edit the ESAPI.properties that your application is going to be using and set the property "Encryptor.CipherTransformation" to a cipher transformation that is FIPS 140-2 certified for you vendor's software. OWASP recommends using a NIST "combined" cipher mode, that is one that provides for both message confidentiality and message authenticity if such a cipher mode is available from your vendor and FIPS 140-2 certified. (Cipher modes "GCM" and "CCM" are the only FIPS 140-2 approved as of this writing.) If such a "combined" cipher mode is not available, then use a cipher transformation like "AES/CBC/PKCS5Padding" with a random IV. This is almost always available as one of the FIPS 140-2 certified cipher transformations, however it will not provide you with message authenticity because to remain FIPS 140-2 compliant you will have had to disable the MAC calculation (via setting Encryptor.CipherText.useMAC to false) causing ESAPI to skip calculating an explicit MAC for you and hence providing no assurance of data integrity to the party attempting to decrypt your data. Without authenticity however, your encryption may still be vulnerable to the "padded oracle" chose ciphertext attack. Consult with your local cryptographic expert in such cases as this depends greatly on the circumstances of how you are using encryption.
5. If your vendor softare requires you to explicitly to initialize their software for FIPS-mode (e.g., to cause the required FIPS 140-2 "power-on self tests" to run) by calling some software method, then this must be done by your application code before calling any of the ESAPI crypto-related code. Specifically, it must be called before your application calls ESAPI.encryptor().

Acknowledgments

In addition, Kevin Kenan has agreed to review all the crypto code from ESAPI 2.0-rc5 or 2.0-rc6 candidate release. Others are invited to participate as well, especially those with a background in cryptography. (See "Request to review ESAPI 2.0 crypto" for details.)

I would also like to thank Jessica Fitzgerald-McKay and Andy Sampson of the Systems and Network Analysis Center of the NSA for their excellent feedback in response to the review request made in Google Issue #81. Working with bureaucracy is not something that I do particularly well, but Jessica and Andy made it as painless as it possibly could be. Their feedback, which at this point, unfortunately we are unable to quote, was generally favorable, but in the places where they had recommendations, these were very precise and helpful.

Lastly, I would like to thank Jeffrey Walton of Software Integrity, LLC. Jeff provided an extremely thorough analysis of ESAPI 2.0's (as of 2.0_rc10) Key Derivation Function (KDF). Jeff's analysis convinced me to bring ESAPI's KDF more in line with NIST's recommendations for KDFs as described in NIST Special Publication 800-108 (and specifically section 5.1). You can read about Jeff's review at Analysis of ESAPI 2.0's Key Derivation Function