Skip to content

Commit 9c8d0ff

Browse files
DAILLYDAILLY
DAILLY
authored and
DAILLY
committed
Nearlly done, final check on github
1 parent 7ae849f commit 9c8d0ff

11 files changed

+286
-42
lines changed

1-CreateKey.md

+69-2
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,80 @@
33
This certificate will be used by the .NET Driver and JDBC Driver using different Key Store.
44

55
Points of the script:
6-
- FriendlyName is mandatory for the JavaKeyStore
6+
- FriendlyName is mandatory for the JavaKeyStore provider.
77
- Exportable is mandatory: you must distribute this certificate to all clients requiring access to encrypted columns.
88
- Set the password for your exported certificate.
99

10+
## Certificate compatibility issue
11+
12+
<table>
13+
<tbody>
14+
<tr>
15+
<td>
16+
<img src="assets\warning.png" alt="warning" width="200"/>
17+
</td>
18+
<td>
19+
This setup used Windows Server 2016 to generate the certificate.
20+
Nevertheless it seems there is an issue in Java to read the certificate generated by this operating system version (at least in the PKCS#12 format (.pfx file).
21+
I used the Oracle JDK 1.8.0_181 and the JDBC driver was unable to read properly the certificate in the JavaKeyStore. This is documented in following bug reports :
22+
<ul>
23+
<li><a href="https://bugs.java.com/view_bug.do?bug_id=JDK-8202299">https://bugs.java.com/view_bug.do?bug_id=JDK-8202299</a></li>
24+
<li><a href="https://bugs.openjdk.java.net/browse/JDK-8202299">https://bugs.openjdk.java.net/browse/JDK-8202299</a></li>
25+
</ul>
26+
These bugs are resolved in the JDK 11 which is not yet publicly available (early access build <a href="http://jdk.java.net/11/">JDK 11</a>. So until general availability you should generate the certificate with:
27+
<ul>
28+
<li>A Windows Server 2012 R2 : nevertheless the New-SelfSignedCertificate cmdlet doesn't present all options (especially the -FriendlyName) so it is not suitable.</li>
29+
<li>A real PKI infrastructure (Windows Server 2012 R2 or below), to create a certificate request with appropriate settings (especially the FriendlyName).</li>
30+
<li>KeyTool.exe available in the Java JDK. This is finally this option that worked for both the .NET Driver and the JDBC Driver.</li>
31+
</td>
32+
</tr>
33+
</tbody>
34+
</table>
35+
36+
37+
## KeyTool
38+
39+
Using keytool.exe (included with JDK) to generate the certificate :
40+
41+
```cmd
42+
c:\Program Files\Java\jdk1.8.0_181\bin>keytool -genkeypair -keyalg RSA -alias CLINIC_CMK_GENERIC -keystore C:\Temp\CLINIC_CMK_GENERIC.pfx -storepass P@ssw0rd -validity 7200 -keysize 4096 -storetype pkcs12 -keypass P@ssw0rd
43+
```
44+
45+
Output with answers to generate the certificate subject :
46+
```txt
47+
What is your first and last name?
48+
[Unknown]: CLINIC_CMK_GENERIC
49+
What is the name of your organizational unit?
50+
[Unknown]: CLINIC
51+
What is the name of your organization?
52+
[Unknown]: CLINIC
53+
What is the name of your City or Locality?
54+
[Unknown]: BOSTON
55+
What is the name of your State or Province?
56+
[Unknown]: IL
57+
What is the two-letter country code for this unit?
58+
[Unknown]: US
59+
Is CN=CLINIC_CMK_GENERIC, OU=CLINIC, O=CLINIC, L=BOSTON, ST=IL, C=US correct?
60+
[no]: yes
61+
```
62+
63+
Import the certificate in the Windows Certificate store for both the .NET Client and Security administrator system. Provide the certificate to the JDBC client as file. Do not provide the certificate to DBA administrator.
64+
All this roles are described in details on [next steps](2.CreateGenericCMK-CEK.md) :
65+
66+
```PowerShell
67+
$pwd = ConvertTo-SecureString -String "P@ssw0rd" -AsPlainText -Force
68+
Import-PfxCertificate -FilePath C:\Temp\CLINIC_CMK_GENERIC.pfx -Exportable -CertStoreLocation Cert:\CurrentUser\My -Password $pwd
69+
```
70+
71+
72+
## PowerShell
73+
74+
With the issue described above, Java clients using a certificate generated with a Windows Server 2016 operating system will not be able to decrypt properly encrypted columns. So this script is just for information purpose until JDK 11 availability.
75+
76+
Windows Server 2006 Certificate generation :
1077
```PowerShell
1178
# Create a new self signed certificate
79+
# FriendlyName is for the JavaKeyStore
1280
$cert = New-SelfSignedCertificate -CertStoreLocation Cert:\CurrentUser\My `
1381
-KeyAlgorithm RSA `
1482
-KeyDescription 'SQL Server Always Encrypted CLINIC CMK' `
@@ -28,5 +96,4 @@ $cert = New-SelfSignedCertificate -CertStoreLocation Cert:\CurrentUser\My `
2896
#export the certificate in a file
2997
$pwd = ConvertTo-SecureString "P@ssw0rd" -AsPlainText -Force
3098
Export-PfxCertificate -Cert Cert:\CurrentUser\my\$($cert.Thumbprint) -FilePath "C:\Temp\CLINIC_CMK_GENERIC.pfx" -Password $pwd
31-
3299
```

2.CreateGenericCMK-CEK.md

+37-6
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,17 @@
22

33
This scenario use the Key provising with role separation provided by [Microsoft](https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/configure-always-encrypted-keys-using-powershell?view=sql-server-2017#KeyProvisionWithRoles).
44

5-
### <span id="setup">Setup environment for Generic Provider usage</span>
5+
There is 2 distinguished roles :
6+
- The security administrator, which has access to the certificate.
7+
- The database (DBA) administrator, which has no access to the certificate.
8+
9+
While this scenario use a complete role separation, for the sake of simplicity everything was performed on the same computer. I just removed the certificate from the Windows Certificate store for the DBA administrator steps.
10+
11+
So I only setup environment once for both.
12+
13+
## <span id="setup">Setup environment for Generic Provider usage</span>
14+
15+
This step must de done only once per computer. This download the PowerShell SqlServer module and overwrite the Microsoft dll with the [patched one](bin\SQLServerAlwaysEncrypted.dll) (making a backup of the originating dll).
616

717
```PowerShell
818
#####################################################
@@ -75,24 +85,32 @@ $customprovider = New-Object -TypeName SqlServerAlwaysEncrypted.SqlColumnEncrypt
7585
#the ProviderName property is a static property ... Microsoft implemented all providers with a static property. So I kept this implementation.
7686
Write-Host $customprovider.MasterKeyPath
7787
Write-Host $([SqlServerAlwaysEncrypted.SqlColumnEncryptionGenericProvider]::ProviderName)
88+
```
7889

90+
Register the generic provider. Be careful with the order of the commands.
91+
Once the SqlServer PowerShell module is imported, not all assemblies are loaded until you use one cmdlet of the module. At this step, no cmdlet from the SqlServer is used yet. The first command "fake" a call to any SqlServer cmdlet to load all the assemblies used by the module. This initialize the internal dictionnay of custom providers.
7992

93+
```PowerShell
8094
<#internally Always Encrypted Microsoft cmdlets do not load directly the [Microsoft.SqlServer.Management.AlwaysEncrypted.Management] assembly.
8195
Call chain:
8296
- [Microsoft.SqlServer.Management.AlwaysEncrypted.Types]::CustomProviders
8397
- [Microsoft.SqlServer.Management.AlwaysEncrypted.Management]::CustomProviders
8498
- [System.Data.SqlClient.SqlConnection]::RegisterColumnEncryptionKeyStoreProviders()
85-
All cmdlets related to CMK/CEK follow this call chain.
86-
All cmdlets related to columns encryption use directly the [System.Data.SqlClient] assembly.
99+
All cmdlets related to CMK / CEK use this call chain and use the [Microsoft.SqlServer.Management.AlwaysEncrypted.Management]::CustomProviders dictionnary.
100+
All cmdlets related to columns encryption use directly the [System.Data.SqlClient] assembly and its internal dictionnary of custom providers.
87101
#>
88102
[Microsoft.SqlServer.Management.AlwaysEncrypted.Types.AlwaysEncryptedManager]::CustomProviders
89103
90-
#Register our custom provider in the correct Always Encrypted dictionnary.
91104
#confirm only one custom provider is registered currently (Azure_Key_Vault)
105+
#PROVIDED CMDLET in the SQLServerAlwaysEncrypted.dll
92106
Get-SqlColumnEncryptionCustomProvider
107+
108+
#Register the generic provider in all custom providers stores (dictionnary)
109+
#PROVIDED CMDLET in the SQLServerAlwaysEncrypted.dll
93110
Register-SqlColumnEncryptionCustomProvider -Provider $customprovider -ProviderName $([SqlServerAlwaysEncrypted.SqlColumnEncryptionGenericProvider]::ProviderName)
94111
95-
#check our custom provider is registered in the different assemblies:
112+
#PROVIDED CMDLET in the SQLServerAlwaysEncrypted.dll
113+
#check our custom provider is registered in the different assemblies
96114
#[Microsoft.SqlServer.Management.AlwaysEncrypted.Management]
97115
Get-SqlColumnEncryptionCustomProvider
98116
@@ -119,8 +137,12 @@ Key Value
119137
AZURE_KEY_VAULT Microsoft.SqlServer.Management.AlwaysEncrypted.AzureKeyVaultProvider.SqlColumnEncryptionAzureKeyVaultProvider
120138
GENERIC SQLServerAlwaysEncrypted.SqlColumnEncryptionGenericProvider
121139
```
140+
<br />
141+
<br />
122142

123-
### CMK Key Exchange
143+
## CMK Key Exchange
144+
145+
This step is done by the security administrator.
124146

125147
```PowerShell
126148
# Create a SqlColumnMasterKeySettings object (the CMK metadata) with information about the generic provider. KeyPath must not be empty. Here i use "NONE", but this may be the certificate thumbprint or whatever you want. This is not important because the generic provider is already configured with the real keypath to the wrapped provider.
@@ -158,6 +180,9 @@ The DBA administrator has no access to the certificate. The DBA administrator mu
158180

159181

160182
```PowerShell
183+
#Import the module
184+
Import-Module "SqlServer"
185+
161186
# Obtain the location of the column master key and the encrypted value of the column encryption key from your Security Administrator, via a CSV file on a share drive.
162187
$keyDataFile = "C:\Temp\ExchangeWithDBAdmin.txt"
163188
$keyData = Import-Csv $keyDataFile
@@ -258,3 +283,9 @@ Set-SqlColumnEncryption -ColumnEncryptionSettings $ces -LogFileDirectory $logdir
258283
9/10/2018 4:13:28 PM INFO MainThread Finalizing data migration.
259284
9/10/2018 4:13:29 PM INFO MainThread Deploying the specified encryption settings completed in 0d:0h:1m:12s.
260285
```
286+
287+
288+
## Clients
289+
Now you can use the client sample to access the database and decrypt values :
290+
- [.NET Client](3-.NETClient.md)
291+
- [JDBC Client](4-JDBCClient.md)

3-.NET Client.md

Whitespace-only changes.

3-.NETClient.md

+83
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# .NET Client SQL Query
2+
3+
The code below represent a SQL query. This is coded in PowerShell but this exactly the same in C#.
4+
- Import the [Generic provider](bin/SQLServerAlwaysEncrypted.dll)
5+
- Create a connectionString (configure with your own settings)
6+
- Create the wrapped provider (MSSQL_CERTIFICATE_STORE)
7+
- Create the generic provider (GENERIC) with the wrapped provider, providing the right path to the certificate for the wrapped provider.
8+
- Create a SQL command with parameters
9+
- Execute Query
10+
- Check the request is successfull and data unencrypted
11+
12+
13+
```PowerShell
14+
#import the extended Always Encrypted types (the Generic provider)
15+
Add-type -Path '.\bin\SqlServerAlwaysEncrypted.dll'
16+
17+
#Configure your connection settings
18+
$constr = New-Object -TypeName System.Data.SqlClient.SqlConnectionStringBuilder
19+
$constr.Item("Data Source") = "192.168.2.19"
20+
$constr.Item("Initial Catalog") = "CLINIC"
21+
$constr.Item("Authentication") = [System.Data.Sqlclient.SqlAuthenticationMethod]::ActiveDirectoryIntegrated
22+
23+
#ensure to setup the Column Encryption Setting
24+
$constr.Item("Column Encryption Setting") = [System.Data.SqlClient.SqlConnectionColumnEncryptionSetting]::Enabled
25+
$constr.Item("TrustServerCertificate") = $true
26+
27+
#access to your certificate from the Windows Certificate Store (take care if there are many to select the right one, here we get only the fisrt)
28+
$cert = Get-childitem -Path (Cert:\CurrentUser\My)[0]
29+
30+
#create the wrapped Windows Certificate Store provider
31+
$sqlstoreprovider = new-object -TypeName System.Data.SqlClient.SqlColumnEncryptionCertificateStoreProvider
32+
33+
#Create the generic provider, providing the inner provider and the right path for this inner provider
34+
$genericProvider = New-Object -TypeName SqlServerAlwaysEncrypted.SqlColumnEncryptionGenericProvider($sqlstoreprovider, "CurrentUser/My/$($cert.Thumbprint)")
35+
36+
#create a dictionnary, fill this dictionnary with the generic provider and register this provider.
37+
$customproviders = New-Object -TypeName 'System.Collections.Generic.Dictionary[String,System.Data.SqlClient.SqlColumnEncryptionKeyStoreProvider]'
38+
$customproviders.Add([SqlServerAlwaysEncrypted.SqlColumnEncryptionGenericProvider]::ProviderName, $genericProvider)
39+
[System.Data.SqlClient.SqlConnection]::RegisterColumnEncryptionKeyStoreProviders($customproviders)
40+
41+
#create the connection and connect
42+
$connection = New-Object System.Data.SqlClient.SqlConnection($constr.ConnectionString);
43+
44+
try
45+
{
46+
$connection.Open();
47+
} catch {
48+
Write-Host "Error connecting"
49+
}
50+
51+
#create the command with parameter @SSN
52+
$command = New-Object -TypeName System.Data.SqlClient.SqlCommand
53+
$command.CommandText = "SELECT [SSN], [FirstName], [LastName], [BirthDate] FROM [dbo].[Patients] WHERE SSN=@SSN";
54+
$command.Connection = $connection
55+
56+
$paramSSN = $command.CreateParameter();
57+
$paramSSN.ParameterName = "@SSN";
58+
$paramSSN.DbType = [System.Data.DbType]::AnsiStringFixedLength;
59+
$paramSSN.Direction = [System.Data.ParameterDirection]::Input;
60+
61+
#provide clear-text value to search in the database. it's the driver responsability to encrypt the value
62+
$paramSSN.Value = "795-73-9838";
63+
$paramSSN.Size = 11;
64+
$command.Parameters.Add($paramSSN);
65+
66+
#execute the query
67+
$reader = $command.ExecuteReader()
68+
69+
if ($reader.HasRows) {
70+
71+
while ($reader.Read())
72+
{
73+
#print result, ensure one entry is returned and the data is in clear-text
74+
[String]::Format("{0}, {1}, {2}, {3}", $reader[0], $reader[1], $reader[2], ([DateTime]$reader[3]).ToShortDateString());
75+
}
76+
}
77+
```
78+
79+
80+
Sample Output :
81+
```
82+
795-73-9838, Catherine, Abel, 9/10/1996
83+
```

4-JDBC Client.md

-4
This file was deleted.

4-JDBCClient.md

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# JDBC Client. SQL Query
2+
3+
The [Java/SQLServerAlwaysEncrypted](JAVA/SQLServerAlwaysEncrypted) project contains both :
4+
- the Generic Provider (wrapping the JAVA_KEY_STORE provider).
5+
- a sample to query the database.
6+
7+
The details of the sample are :
8+
- get a connectionString (from arguments)
9+
- Create the wrapped provider (JAVA_KEY_STORE) with right settings.
10+
- Create the generic provider (GENERIC) with the wrapped provider, providing the right path for the wrapped provider.
11+
- Create a SQL command with parameters
12+
- Execute Query
13+
- Check the request is successfull and data unencrypted
14+
15+
16+
The JAVA_KEY_STORE needs 2 parameters in its constructor (static String in code) :
17+
- KeyStorePath : the path to the certificate on the file system (.pfx file)
18+
- KeyStorePassword : the password to open the key store (see [Create Key](1-CreateKey.md))
19+
20+
The CMK does not provide any valid value for the key path. The JAVA_KEY_STORE require this path being the FriendlyName property of the certificate.
21+
22+
Create an instance of the GENERIC_KEY_STORE, providing both the wrapped JAVA_KEY_STORE instance and the friendlyname of the certificate as path ("clinic_cmk_generic" in the [sample](1-CreateKey.md)).
23+
24+
To keep authentication simple with this sample, I created a new account (admin P@ssw0rd) in SSMS.
25+
26+
Provide the connectionstring as arguments (if you use the sample database CLINIC) :
27+
- exemple in Eclipse Debug Configuration :
28+
29+
![Eclipse Debug arguments](assets/jdbc_eclipse_debug_connectionstring.png)
30+
31+
Else you can inspire from source code to create you own SQL query. The source code is really easy. Everything is detailed enough.
32+
33+
Sample Output :
34+
```
35+
Connecting to SQL Server ... Done.
36+
Executing SQL Query ...
37+
SSN: 795-73-9838, FirstName: Catherine, LastName:Abel, Date of Birth: 1996-09-10
38+
Done.
39+
```

Issue1.md

+6-3
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
# PS Module unable to retrieve a registered custom provider
1+
# SqlServer PowerShell Module unable to find a registered custom provider
2+
23

34
## Microsoft Code
45
Initial code from the Microsoft.SqlServer.Management.AlwaysEncrypted.Management.dll:
@@ -32,11 +33,13 @@ public static class AlwaysEncryptedManagement {
3233

3334
The main issue in this code is the default statement. Only an error is returned whenever you attempt to access a custom provider. Normally the code should behave as in the "AZURE_KEY_VAULT" statement.
3435

35-
With the original code, it is impossible to achieve Always Encrypted configuration with PowerShell and a custom/generic provider. It may be possible with a full C# or Java, but all the documentation use PowerShell.
36+
With the original code, it is impossible to achieve Always Encrypted configuration with PowerShell and a custom/generic provider. It may probably be possible with a full C# or Java, but all the documentation used PowerShell.
37+
38+
In an effort to use PowerShell for this scenario, a solution was necessary.
3639

3740
## Patched DLL
3841

39-
The [provided patched DLL](bin/Microsoft.SqlServer.Management.AlwaysEncrypted.Management.dll) replace the Microsoft original code with the following simple code. I was not really able to properly insert IL to replace only the default statement from the original code. But it doesn't have any importance, being able to retrieve a custom/generic provider is sufficient.
42+
The [provided patched DLL](bin/Microsoft.SqlServer.Management.AlwaysEncrypted.Management.dll) replace the Microsoft original code with the following simple code. I was not really able to properly insert IL code to replace only the default statement from the original code. But it doesn't have any importance, being able to retrieve a custom/generic provider is sufficient.
4043

4144
I used ILSpy + Reflexil to achieve this.
4245

Issue2.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Information about the issue
44

5-
One of the issue encountered during development of this generic provider was the moment where you want to encrypt columns. The cmdlet used to encrypt the columns does use the [System.Data.SqlClient.SqlConnection] object to get access to registered custom providers.
5+
One of the issue encountered during development of this generic provider was the moment where you want to encrypt columns. The cmdlet used to encrypt the columns use the [System.Data.SqlClient.SqlConnection] object to get access to registered custom providers.
66

77
By following the <a href="#callstack">callstack</a> from the load of the PowerShell module, it is clear that both the PowerShell module and the code from [System.Data.SqlClient] present some issues:
88
- The SqlServer PowerShell module loads the "AZURE_KEY_VAULT" as a custom provider and by doing so prevent any other registration of a custom provider.

0 commit comments

Comments
 (0)