Para a primeira parte em descriptografar o segredo pré mestre do servidor, tenho usado:
openssl rsautl -in cpre.key -inkey key.pem -decrypt -out spre.key
onde cpre.key está em binário quando alimentado para isso e key.pem é minha chave privada gerada ao fazer meu certificado. Compartilharei minha chave privada aqui porque não usarei isso, exceto em minha prova de conceito.
key.pem
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDDTYy4zhRmPYAS
qpnpbHzHNgTtg6Cn1hGGuYzeoIvbewd/HfwmTYlnnYJNtZESkZgy09uci+B8d3T1
it4WAkJv8j6QCg4PHzmPovsdvKuF8WUrhZsmiNk8K+Hn7fkNCqbK/NsghOi+RwUQ
JCudz6OsYT/jqx4TGCOqEpqWF4ShBdsUJccPxkpXn/KZn330pGJTVkVCAfae/btA
Ylg8QY0/LNkbvkp+2UlglyV8DMqWo125GEnD56Aphrb0aMdTC0/tlI9oQWSpWhT9
IeJtR7vsQLSVmscotalktuCFTaaP6NzLg5rVvJAmxurJJTETafXB4rb55OWpzyij
t6qcwUkdAgMBAAECggEAUpCkXzRwF1EEpd+NtcrxbV5+8tJkxs+PFT+4OB3zQlRe
tPS+Noss/aru/V8293yptUMvYp0XyEbQIUYV4YQPBjiJDmfRzb7LnNewBcBPWw7U
UHPpRX8xJ2j91gka8tJ/9Oy0zqihpF9t2ocQEWVIsoVidyNOUZAw86NtyuoiXdfY
pgJxTVdCH0QjYb7jIQAWIhJXvp8DVLFRn42lZD96YS0GJyBOtXg+RXXUSWrVUcjb
Gi8w7eRbLKbGtYZ7prTBeoYubFuuyML4IF/MwkMBbjU3sJCV4Q6F2R1Kjv4RMFge
oqyf+YDq26USUgdIVQJaUPj8g98qvS8jCH+OTVkIGQKBgQDqHMgPREuMFaqihi9p
AqcWDvUI+gLkDAHuIr5WTLFNEt6SFh5vVtiaVPgANNqR5FSPPczvOF8WGuU6zapf
wbuyqzPCVuxfcllR0DCVP2J2VM14WX2D9VUqPRNXM4shQnvZh/jbhgbqIfWdLuAq
cXAJ+O2FzGUmPWUexsR5OyZVrwKBgQDVj+gG2myl6juC6jBDXhD359QuZN+L6jhy
6HZyPPxQgMz7Flgn+oifiysAsfY6uTxyrt5eSVO23HoqfI1/C12xtF1H+LdSdMO7
60jikauByZRoSuVDxuDsMFBTOf018lO5MI03tKy4LeFYCJQ1VA8t0cK1zInn7YiL
7UyScnJM8wKBgQDYNkiVwO88qu3yN5cllmuVta6b4bo1vtkxgyjiLuY4Vkr0BDWL
glt8/SglmKqbvpJR0DOqYxd+JMy76CGHWjbtJawhuYGIDOVs56dWzzkyK9OvaIbB
Dhzfa5SnHlVKHEqhdjBhHYDZTw+1YQtqZ2Wf8CP9Q+3OyTjQmaXuiLA4xQKBgDvl
sbECYAikym4sWr76Sg4NwnAEsMIgPhbUZiwiOK4ngdZGbRFBbkI5GSWHxESyIWcU
7p9JN6aI7eauqkzgljW6HZOLCXeh8ifplLafHpg3Pl2F7Al3ZSw5XL4K/J69yYee
O21NSkgZ7iLvvQm7+sniFBdDN8aJ04tTUe+dE7vLAoGAHcY/RslRZ1jusaEY8xPW
7BMzcPLhvbeoMFKE4ruryExNCnZQ4/9jKT9S0IXk1L8peOl8TiK1TFeEeFmijFG/
qqPgglcoJu0kscNQmVmt0JgLz6FyB1+oP4faKWtW7GI+WdGXcvAcUgaDMVrPEN0u
9wnF7LizmwRzYywT3NFWhb4=
-----END PRIVATE KEY-----
cpre.key (hex) capturou cliente criptografado pré mestre
b14ebd6c882b62bd2a527763b24eec0159694b57d50c091625797cc348ecbde551a6b5bc17a676f2
a9b1a16a7f0a8694fd3be297ffcd5189baa2cfc6ae3a21e608ebc2dfc40d11a833445ff3955fb230
dfefe6c741b2d48bf2977f3b3ba9ccc6fa0069a59c7d48579a3eb43562a1aeb7312857b7dede6b3a
e2f1e80390697518b636a064e37c5a3bb696f4af98c38661995695e536a7847d9b3ae4e0ee943da8
f6e577c6e1d7e1f30e9b67f62a0301a4fb6fc401341c3fb5f3a00c56d67c7d26a31a5a462a0e3610
f6de24f95812e05ca6c5916d89573429eab173595d4598fcad39ef7bff3b07d74cdc6928990fe4ac
80db36b7fa00bfe5534de164e9776316
spre.key (hex) descriptografado do lado do servidor pré mestre
03033ab0d673caefc4a9f6d35cf51d349815d67a09ac454f11dfb9741c96001a66a3e82f688b8213
8403018384cf35f2
Dave, percebo que você disse que era improvável que eu estivesse descriptografando isso incorretamente, mas você vê algum problema no meu primeiro passo? Eu posso estar capturando os dados do cliente incorretamente, então eu queria verificar.
Tags tls openssl encryption public-key rsa