Microsoft Outlook: Control the encryption algorithm used for S/MIME.

When S/MIME certificates are issued, they usually contain a certificate extension "S/MIME Capabilities". This certificate extension is specified in RFC 4262 and can be used by compatible e-mail programs to specify the symmetric algorithms supported by the recipient of an encrypted message. The sender should then choose the strongest algorithm supported by the recipient.

Microsoft Outlook uses (if available and required) the information in the "S/MIME Capabilities" extension of a certificate. Below is a description of how it is used and which algorithms are selected.

How does Outlook behave when the extension is present and how when it is not?

Outlook behaves as follows, depending on whether the certificate extension is present or not:

• If S/MIME Capabilities certificate extension is present in the recipient's certificate (or the information is available due to previous receipt of a signed email), the strongest algorithm from the list is selected and used.
• If S/MIME Capabilities certificate extension in the recipient's certificate not is present, the default algorithm is configured and used

The default algorithm differs depending on the Outlook version used:

• Outlook from 2016 with hotfix KB4484511 and Outlook 2019 uses AES-256 as the default algorithm.
• Outlook from 2010 with Hotfix Package from 22.02.2011 up to Outlook 2016 without hotfix KB4484511 uses 3DES as the default algorithm.
• Outlook until 2010 with Hotfix package from 22.02.2011 use RC2-40 as the default algorithm.

Override Outlook behavior

The default algorithm can be customized using the following registry key.

HKEY_CURRENT_USER\SOFTWARE\Policies\Microsoft\office\16.0\outlook\security

If the two registry values mentioned above are configured, the "S/MIME Capabilities" (no matter whether received via a signed e-mail or present as a certificate extension) are ignored and the values configured here are permanently used.

Value	Type	Description
UseAlternateDefaultEncryptionAlg	REG_DWORD	Overrides the settings in the Outlook Trust Center. The encryption algorithm set in the DefaultEncryptionAlgOID value is used.
DefaultEncryptionAlgOID	REG_SZ	Determines the encryption algorithm to be used. It must be stored in the form of an OID (see table).

Possible values are:

OID	Description
1.2.840.113549.3.7	3DES in CBC mode with 168 bit key length
2.16.840.1.101.3.4.1.2	AES in CBC mode with 128 bit key length
2.16.840.1.101.3.4.1.22	AES in CBC mode with 192 bit key length
2.16.840.1.101.3.4.1.42	AES in CBC mode with 256 bit key length

Related links:

• The "S/MIME Capabilities" certificate extension

External sources

• September 1, 2020, update for Outlook 2016 (KB4484511) (Microsoft)
• Description of the Outlook 2010 hotfix package (outlook-x-none.msp): February 22, 2011 (Microsoft)
• Your Digital ID name cannot be found error when decrypting a message by using a 3DES certificate (Microsoft)
• Plan for e-mail messaging cryptography in Outlook 2010 (Microsoft)
• When using Fast Smartcard and attempting to decrypt an e-mail encrypted using AES256 encryption in Outlook you are presented with the following error: "Sorry were having trouble opening this item...". (Citrix)