Feature request: OpenSSL::PKey::PKey, OpenSSL::PKey::ML-DSA, or OpenSSL::PKey::ML-DSA-NN to implement #private? and #to_pem #1085

Activity

1. rhenium commented on Jul 15, 2026

   

   rhenium

   on Jul 15, 2026

   Member

   More actions

   Implementing #private? on the base class requires ruby/openssl to keep track of which components an OpenSSL::PKey::PKey contains, because OpenSSL doesn't provide a generic public API to check whether an EVP_PKEY is a private key, a public key, or key parameters only (openssl/openssl#9467).

   I looked into this before in #527, but eventually gave up because supporting a wide range of OpenSSL versions seemed to make it too fragile and complicated. I think we could revisit this after dropping support for OpenSSL 1.1.1 (IOW, after removing OpenSSL::PKey::{RSA,DSA,DH}#set_* methods).

   {RSA,DSA,DH,EC}#to_pem uses several different non-standard formats (which OpenSSL's man pages refer to as traditional). PKey#private_to_pem always uses PKCS#8 regardless of the pkey type. My recommendation is to use #private_to_pem whenever possible, though you might need to continue using #to_pem on RSA/DSA/DH/EC for backwards compatibility.

   Or I want OpenSSL::PKey::ML-DSA or OpenSSL::PKey::ML-DSA-NN classes to have the missing methods.

   FWIW the reason we haven't added dedicated classes for ML-DSA, ML-KEM, Ed25519, X25519, etc. is simply that we haven't needed to introduce methods specific to these pkey types.
2. junaruga commented on Jul 16, 2026

   

   junaruga

   on Jul 16, 2026

   MemberAuthor

   More actions

   Thanks for the explaining the context.

   For #private?, I experimented the following method for ML-DSA-44 with Ruby compiled with OpenSSL 4.1 dev.

   diff --git a/lib/openssl/pkey.rb b/lib/openssl/pkey.rb
   index 39871e1..9a9d43a 100644
   --- a/lib/openssl/pkey.rb
   +++ b/lib/openssl/pkey.rb
   @@ -7,6 +7,15 @@
    require_relative 'marshal'
   
    module OpenSSL::PKey
   +  class PKey
   +    def private?
   +      private_to_pem
   +      true
   +    rescue OpenSSL::PKey::PKeyError
   +      false
   +    end
   +  end
   +
      # Alias of PKeyError. Before version 4.0.0, this was a subclass of PKeyError.
      DHError = PKeyError

   test.rb

   require 'openssl'
   
   private_key = OpenSSL::PKey.generate_key("ML-DSA-44")
   p private_key.private?
   public_key = OpenSSL::PKey.read(private_key.public_to_der)
   p public_key.private?

   $ which ruby
   ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/bin/ruby
   

   $ ruby -I lib/ test.rb
   true
   false
   

   The test.rb worked as I expected in OpenSSL 4.1 dev. This behavior depends on the PEM_write_bio_PKCS8PrivateKey returning 0 when the argument pkey is public key in my case. But this situation is fragile when considering other OpenSSL versions and other SSL library LibreSSL and AWS-LC, right?

   openssl/ext/openssl/ossl_pkey.c

   Lines 966 to 970 in 27c59a4

   	if (!PEM_write_bio_PKCS8PrivateKey(bio, pkey, enc, NULL, 0,
   	ossl_pem_passwd_cb, (void *)pass)) {
   	BIO_free(bio);
   	ossl_raise(ePKeyError, "PEM_write_bio_PKCS8PrivateKey");
   	}

   For the #to_pem, Yes, the reason I keep using #to_pem is because of the backward compatibility.
   But I think I need to refactor the current existing #to_pem to replace it with #public_to_pem or #private_to_pem in ruby/rubygems before working on ruby/rubygems#9697.
3. rhenium commented on Jul 16, 2026

   

   rhenium

   on Jul 16, 2026

   Member

   More actions

   I don't think we can implement a generic #private? this way in ruby/openssl. Although ruby/openssl currently has little support for third-party providers, a pkey backed by an HSM-based provider wouldn't allow exporting the private key with #private_to_pem.
4. junaruga

   added 2 commits that reference this issue on Jul 17, 2026

   Remove #private? check from open_private_key and check in Gem::Securi…

   ...

   017733a

   Remove #private? check from open_private_key and check in Gem::Securi…

   ...

   cef0fe7
5. junaruga commented on Jul 30, 2026

   

   junaruga

   on Jul 30, 2026

   Last edited by junaruga

   MemberAuthor

   More actions

   Thanks for your comment. I see the best practice to deal with OpenSSL::PKey::PKey and OpenSSL::PKey::{RSA,DSA,EC} together is to stop using the OpenSSL::PKey::{RSA,DSA,EC}#to_pem and #private? according to ruby/rubygems#9701. I would close this ticket.
6. junaruga

   closed this as completedon Jul 30, 2026
7. junaruga

   closed this as not plannedon Jul 30, 2026
8. rhenium commented on Aug 1, 2026

   

   rhenium

   on Aug 1, 2026

   Member

   More actions

   I think it would still be nice to have, but we'll have to wait for the underlying C API to be available.
9. matzbot

   added a commit that references this issue on Aug 3, 2026

   [ruby/rubygems] Remove #private? check from open_private_key and chec…

   ...

   53b7087
10. junaruga commented on Aug 3, 2026

    

    junaruga

    on Aug 3, 2026

    MemberAuthor

    More actions

    Yes, I would agree on your opinion.