Introduction
In this post, we will see
- Python 3.9.x patch for FIPS enabled Openssl
- Test with Hashlib
- Test with Cryptography module
- Test with libcrypto shared library
In previous post, we saw how we built FIPS-enabled Openssl.
It is important to note that even you have FIPS enabled Openssl, still you need something to invoke this. Only setting to environment variable will not work if you are working from other language like Python. So, you need those methods in Python to get/set fips mode in Openssl.
Python 3.9.x patch
Note: This patch is tested for Python 3.9.0 and 3.9.2
This patch is built over https://bugs.python.org/issue27592
The older patch was exposing two methods FIPS_mode() and FIPS_mode_set() in Python.
Python 3.9 solves the issue when even your Openssl is fips enabled. Still, you are able to get md5. We solved this issue in Python 3.7.9 patch for FIPS
diff -aur Python-3.9.0__orig/Lib/ssl.py Python-3.9.0/Lib/ssl.py
--- Python-3.9.0__orig/Lib/ssl.py 2020-10-05 15:07:58.000000000 +0000
+++ Python-3.9.0/Lib/ssl.py 2021-03-02 04:23:32.026226000 +0000
@@ -111,6 +111,11 @@
# LibreSSL does not provide RAND_egd
pass
+try:
+ from _ssl import FIPS_mode, FIPS_mode_set
+except ImportError as e:
+ sys.stderr.write('error in importing\n')
+ sys.stderr.write(str(e))
from _ssl import (
HAS_SNI, HAS_ECDH, HAS_NPN, HAS_ALPN, HAS_SSLv2, HAS_SSLv3, HAS_TLSv1,
diff -aur Python-3.9.0__orig/Modules/Setup Python-3.9.0/Modules/Setup
--- Python-3.9.0__orig/Modules/Setup 2020-10-05 15:07:58.000000000 +0000
+++ Python-3.9.0/Modules/Setup 2021-03-02 04:24:28.071717000 +0000
@@ -207,14 +207,14 @@
#_csv _csv.c
# Socket module helper for socket(2)
-#_socket socketmodule.c
+_socket socketmodule.c
# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
-#SSL=/usr/local/ssl
-#_ssl _ssl.c \
-# -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-# -L$(SSL)/lib -lssl -lcrypto
+SSL=/usr/local/ssl
+_ssl _ssl.c \
+ -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
+ -L$(SSL)/lib -lssl -lcrypto
# The crypt module is now disabled by default because it breaks builds
# on many systems (where -lcrypt is needed), e.g. Linux (I believe).
diff -aur Python-3.9.0__orig/Modules/_ssl.c Python-3.9.0/Modules/_ssl.c
--- Python-3.9.0__orig/Modules/_ssl.c 2020-10-05 15:07:58.000000000 +0000
+++ Python-3.9.0/Modules/_ssl.c 2021-03-02 04:25:30.930669000 +0000
@@ -5394,6 +5394,20 @@
return PyLong_FromLong(RAND_status());
}
+static PyObject *
+_ssl_FIPS_mode_impl(PyObject *module) {
+ return PyLong_FromLong(FIPS_mode());
+}
+
+static PyObject *
+_ssl_FIPS_mode_set_impl(PyObject *module, int n) {
+ if (FIPS_mode_set(n) == 0) {
+ _setSSLError(ERR_error_string(ERR_get_error(), NULL) , 0, __FILE__, __LINE__);
+ return NULL;
+ }
+ Py_RETURN_NONE;
+}
+
#ifndef OPENSSL_NO_EGD
/* LCOV_EXCL_START */
/*[clinic input]
@@ -5875,6 +5889,8 @@
_SSL_ENUM_CRLS_METHODDEF
_SSL_TXT2OBJ_METHODDEF
_SSL_NID2OBJ_METHODDEF
+ _SSL_FIPS_MODE_METHODDEF
+ _SSL_FIPS_MODE_SET_METHODDEF
{NULL, NULL} /* Sentinel */
};
diff -aur Python-3.9.0__orig/Modules/clinic/_ssl.c.h Python-3.9.0/Modules/clinic/_ssl.c.h
--- Python-3.9.0__orig/Modules/clinic/_ssl.c.h 2020-10-05 15:07:58.000000000 +0000
+++ Python-3.9.0/Modules/clinic/_ssl.c.h 2021-03-02 04:27:06.120295000 +0000
@@ -1204,6 +1204,45 @@
return _ssl_RAND_status_impl(module);
}
+PyDoc_STRVAR(_ssl_FIPS_mode__doc__,
+"FIPS Mode");
+
+#define _SSL_FIPS_MODE_METHODDEF \
+ {"FIPS_mode", (PyCFunction)_ssl_FIPS_mode, METH_NOARGS, _ssl_FIPS_mode__doc__},
+
+static PyObject *
+_ssl_FIPS_mode_impl(PyObject *module);
+
+static PyObject *
+_ssl_FIPS_mode(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+ return _ssl_FIPS_mode_impl(module);
+}
+
+PyDoc_STRVAR(_ssl_FIPS_mode_set_doc__,
+"FIPS Mode Set");
+
+#define _SSL_FIPS_MODE_SET_METHODDEF \
+ {"FIPS_mode_set", (PyCFunction)_ssl_FIPS_mode_set, METH_O, _ssl_FIPS_mode_set_doc__},
+
+static PyObject *
+_ssl_FIPS_mode_set_impl(PyObject *module, int n);
+
+static PyObject *
+_ssl_FIPS_mode_set(PyObject *module, PyObject *arg)
+{
+ PyObject *return_value = NULL;
+ int n;
+
+ if (!PyArg_Parse(arg, "i:FIPS_mode_set", &n)) {
+ goto exit;
+ }
+ return_value = _ssl_FIPS_mode_set_impl(module, n);
+
+exit:
+ return return_value;
+}
+
#if !defined(OPENSSL_NO_EGD)
PyDoc_STRVAR(_ssl_RAND_egd__doc__,
How to apply Patch
Download Python 3.9.2 source code from https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz
wget https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz
tar -xzf Python-3.9.2.tgz
cd Python-3.9.2
Assuming patch file patch.diff is placed outside Python folder.
patch -p1 < ../patch.diff Compile Python
Now comes the tricker part. We need to carefully set the compiler flags and linker flags.
CFLAGS=-Wl,--enable-new-dtags,-rpath,/usr/local/ssl/lib LDFLAGS=-L/usr/local/ssl/lib CPPFLAGS=-I/usr/local/ssl/include LIBS=-lcrypto ./configure --enable-shared
make
make installIf above steps goes well. Python is installed on your machine.
Test Python
export LD_LIBRARY_PATH=/usr/local/ssl/lib:/usr/local/libRun version command,
python3.9 --version
Python 3.9.2Build Cryptography Module
Although, our work is complete with basic Python/hashlib. But, if somebody installed another python module cryptography, he/she can still able to calculate unsupported FIPS-encyption algorithm like md5.
Its a little complex trick, where we need to install cryptography module in a different way.
# upgrade pip
python3.9 -m pip install --upgrade pip
# install wheel
python3.9 -m pip install wheel
# Build cryptography wheel
CRYPTOGRAPHY_DONT_BUILD_RUST=1 CFLAGS="-I/usr/local/ssl/include" LDFLAGS="-L/usr/local/ssl/lib" python3.9 -m pip wheel --no-binary :all: cryptography==3.0
# This will create a wheel file in current directory with name: cryptography-3.0-cp39-cp39-linux_x86_64.whl
# install cryptography module now
CRYPTOGRAPHY_DONT_BUILD_RUST=1 CFLAGS="-I/usr/local/ssl/include" LDFLAGS="-L/usr/local/ssl/lib" python3.9 -m pip install cryptography-3.0-cp39-cp39-linux_x86_64.whl Note: We are installing cryptography-3.0, not the latest one. As, it does not have support for our FIPS enabled Openssl (1.0.2t)
We have solved our most of the problem, now it the time to test things.
Testing - via Hashlib
import ssl
print(ssl.OPENSSL_VERSION)
print(ssl.FIPS_mode())
ssl.FIPS_mode_set(1)
print(ssl.FIPS_mode())
import hashlib
print(hashlib.sha1("test_str".encode('utf-8')).hexdigest())
print(hashlib.md5("test_str".encode('utf-8')).hexdigest())Output
OpenSSL 1.0.2t-fips 10 Sep 2019
0
1
f9a90e7c1ff51236191623b84267d110c617118a
Traceback (most recent call last):
File "/python_downloads/Python-3.9.2/t1.py", line 12, in <module>
print(hashlib.md5("test_str".encode('utf-8')).hexdigest())
ValueError: [digital envelope routines: FIPS_DIGESTINIT] disabled for fipsTesting - via libcrypto
import sys
import ssl
import ctypes
libcrypto = ctypes.CDLL("libcrypto.so.1.0.0")
fips_mode = libcrypto.FIPS_mode
fips_mode.argtypes = []
fips_mode.restype = ctypes.c_int
fips_mode_set = libcrypto.FIPS_mode_set
fips_mode_set.argtypes = [ctypes.c_int]
fips_mode_set.restype = ctypes.c_int
text = b""
if __name__ == "__main__":
print("OPENSSL_VERSION: {:s}".format(ssl.OPENSSL_VERSION))
print("FIPS_mode(): {:d}".format(fips_mode()))
print("FIPS_mode_set(1): {:d}".format(fips_mode_set(1)))
print("FIPS_mode(): {:d}".format(fips_mode()))
import hashlib
print("SHA1: {:s}".format(hashlib.sha1(text).hexdigest()))
print("MD5: {:s}".format(hashlib.md5(text).hexdigest()))Output
OPENSSL_VERSION: OpenSSL 1.0.2t-fips 10 Sep 2019
FIPS_mode(): 0
FIPS_mode_set(1): 1
FIPS_mode(): 1
SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
Traceback (most recent call last):
File "/python_downloads/Python-3.9.2/t1.py", line 26, in <module>
print("MD5: {:s}".format(hashlib.md5(text).hexdigest()))
ValueError: [digital envelope routines: FIPS_DIGESTINIT] disabled for fipsTesting - via Cryptography Hazmat package
As I mentioned before, cryptography module is a dangerous one, as its package hazmat called Hazardous Materials provides access to lower level library where you can talk directly to openssl.
from cryptography.hazmat.backends.openssl.backend import backend
import cffi
ffi = cffi.FFI()
lib = backend._lib
print(lib.FIPS_mode())
lib.FIPS_mode_set(1)
print(lib.FIPS_mode())
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.backends import default_backend
digest = hashes.Hash(hashes.MD5(), default_backend())
digest.update(b"abc")
digest.update(b"123")
print(digest.finalize())Output
0
1
Traceback (most recent call last):
File "/python_downloads/Python-3.9.2/t1.py", line 12, in <module>
digest = hashes.Hash(hashes.MD5(), default_backend())
File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/primitives/hashes.py", line 85, in __init__
self._ctx = self._backend.create_hash_ctx(self.algorithm)
File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 342, in create_hash_ctx
return _HashContext(self, algorithm)
File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/backends/openssl/hashes.py", line 36, in __init__
self._backend.openssl_assert(res != 0)
File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 234, in openssl_assert
return binding._openssl_assert(self._lib, ok)
File "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 69, in _openssl_assert
raise InternalError(
cryptography.exceptions.InternalError: Unknown OpenSSL error. This error is commonly encountered when another library is not cleaning up the OpenSSL error stack. If you are using cryptography with another library that uses OpenSSL try disabling it before reporting a bug. Otherwise please file an issue at https://github.com/pyca/cryptography/issues with information on how to reproduce this. ([_OpenSSLErrorWithText(code=101351587, lib=6, func=168, reason=163, reason_text=b'error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips')])How to use Md5 if you really have to
Python3.9 has really started work towards FIPS. They have exposed a flag usedforsecurity in each hashing function via hashlib. Default value is True
Example
import ssl
print(ssl.OPENSSL_VERSION)
print(ssl.FIPS_mode())
ssl.FIPS_mode_set(1)
print(ssl.FIPS_mode())
import hashlib
print(hashlib.md5("test_str".encode('utf-8'), usedforsecurity=False).hexdigest())You might question, I have enabled FIPS. Then why the hell, md5 still worked. Well, there are many situations where you would want to calculate md5, as its fast. And, many libraries where you might be checking for file changed or not, has support for only md5. So, you can not get away with md5.
The objective is that developer knows that he is using these algorithms and it is not for security.
Let me know if you have any query.
