1:   ENGINE
    2:   ======
    3: 
    4:   With OpenSSL 0.9.6, a new component was added to support alternative
    5:   cryptography implementations, most commonly for interfacing with external
    6:   crypto devices (eg. accelerator cards). This component is called ENGINE,
    7:   and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases)
    8:   caused a little confusion as 0.9.6** releases were rolled in two
    9:   versions, a "standard" and an "engine" version. In development for 0.9.7,
   10:   the ENGINE code has been merged into the main branch and will be present
   11:   in the standard releases from 0.9.7 forwards.
   12: 
   13:   There are currently built-in ENGINE implementations for the following
   14:   crypto devices:
   15: 
   16:       o CryptoSwift
   17:       o Compaq Atalla
   18:       o nCipher CHIL
   19:       o Nuron
   20:       o Broadcom uBSec
   21: 
   22:   In addition, dynamic binding to external ENGINE implementations is now
   23:   provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE"
   24:   section below for details.
   25: 
   26:   At this stage, a number of things are still needed and are being worked on:
   27: 
   28:       1 Integration of EVP support.
   29:       2 Configuration support.
   30:       3 Documentation!
   31: 
   32: 1 With respect to EVP, this relates to support for ciphers and digests in
   33:   the ENGINE model so that alternative implementations of existing
   34:   algorithms/modes (or previously unimplemented ones) can be provided by
   35:   ENGINE implementations.
   36: 
   37: 2 Configuration support currently exists in the ENGINE API itself, in the
   38:   form of "control commands". These allow an application to expose to the
   39:   user/admin the set of commands and parameter types a given ENGINE
   40:   implementation supports, and for an application to directly feed string
   41:   based input to those ENGINEs, in the form of name-value pairs. This is an
   42:   extensible way for ENGINEs to define their own "configuration" mechanisms
   43:   that are specific to a given ENGINE (eg. for a particular hardware
   44:   device) but that should be consistent across *all* OpenSSL-based
   45:   applications when they use that ENGINE. Work is in progress (or at least
   46:   in planning) for supporting these control commands from the CONF (or
   47:   NCONF) code so that applications using OpenSSL's existing configuration
   48:   file format can have ENGINE settings specified in much the same way.
   49:   Presently however, applications must use the ENGINE API itself to provide
   50:   such functionality. To see first hand the types of commands available
   51:   with the various compiled-in ENGINEs (see further down for dynamic
   52:   ENGINEs), use the "engine" openssl utility with full verbosity, ie;
   53:        openssl engine -vvvv
   54: 
   55: 3 Documentation? Volunteers welcome! The source code is reasonably well
   56:   self-documenting, but some summaries and usage instructions are needed -
   57:   moreover, they are needed in the same POD format the existing OpenSSL
   58:   documentation is provided in. Any complete or incomplete contributions
   59:   would help make this happen.
   60: 
   61:   STABILITY & BUG-REPORTS
   62:   =======================
   63: 
   64:   What already exists is fairly stable as far as it has been tested, but
   65:   the test base has been a bit small most of the time. For the most part,
   66:   the vendors of the devices these ENGINEs support have contributed to the
   67:   development and/or testing of the implementations, and *usually* (with no
   68:   guarantees) have experience in using the ENGINE support to drive their
   69:   devices from common OpenSSL-based applications. Bugs and/or inexplicable
   70:   behaviour in using a specific ENGINE implementation should be sent to the
   71:   author of that implementation (if it is mentioned in the corresponding C
   72:   file), and in the case of implementations for commercial hardware
   73:   devices, also through whatever vendor support channels are available.  If
   74:   none of this is possible, or the problem seems to be something about the
   75:   ENGINE API itself (ie. not necessarily specific to a particular ENGINE
   76:   implementation) then you should mail complete details to the relevant
   77:   OpenSSL mailing list. For a definition of "complete details", refer to
   78:   the OpenSSL "README" file. As for which list to send it to;
   79: 
   80:      openssl-users: if you are *using* the ENGINE abstraction, either in an
   81:           pre-compiled application or in your own application code.
   82: 
   83:      openssl-dev: if you are discussing problems with OpenSSL source code.
   84: 
   85:   USAGE
   86:   =====
   87: 
   88:   The default "openssl" ENGINE is always chosen when performing crypto
   89:   operations unless you specify otherwise. You must actively tell the
   90:   openssl utility commands to use anything else through a new command line
   91:   switch called "-engine". Also, if you want to use the ENGINE support in
   92:   your own code to do something similar, you must likewise explicitly
   93:   select the ENGINE implementation you want.
   94: 
   95:   Depending on the type of hardware, system, and configuration, "settings"
   96:   may need to be applied to an ENGINE for it to function as expected/hoped.
   97:   The recommended way of doing this is for the application to support
   98:   ENGINE "control commands" so that each ENGINE implementation can provide
   99:   whatever configuration primitives it might require and the application
  100:   can allow the user/admin (and thus the hardware vendor's support desk
  101:   also) to provide any such input directly to the ENGINE implementation.
  102:   This way, applications do not need to know anything specific to any
  103:   device, they only need to provide the means to carry such user/admin
  104:   input through to the ENGINE in question. Ie. this connects *you* (and
  105:   your helpdesk) to the specific ENGINE implementation (and device), and
  106:   allows application authors to not get buried in hassle supporting
  107:   arbitrary devices they know (and care) nothing about.
  108: 
  109:   A new "openssl" utility, "openssl engine", has been added in that allows
  110:   for testing and examination of ENGINE implementations. Basic usage
  111:   instructions are available by specifying the "-?" command line switch.
  112: 
  113:   DYNAMIC ENGINES
  114:   ===============
  115: 
  116:   The new "dynamic" ENGINE provides a low-overhead way to support ENGINE
  117:   implementations that aren't pre-compiled and linked into OpenSSL-based
  118:   applications. This could be because existing compiled-in implementations
  119:   have known problems and you wish to use a newer version with an existing
  120:   application. It could equally be because the application (or OpenSSL
  121:   library) you are using simply doesn't have support for the ENGINE you
  122:   wish to use, and the ENGINE provider (eg. hardware vendor) is providing
  123:   you with a self-contained implementation in the form of a shared-library.
  124:   The other use-case for "dynamic" is with applications that wish to
  125:   maintain the smallest foot-print possible and so do not link in various
  126:   ENGINE implementations from OpenSSL, but instead leaves you to provide
  127:   them, if you want them, in the form of "dynamic"-loadable
  128:   shared-libraries. It should be possible for hardware vendors to provide
  129:   their own shared-libraries to support arbitrary hardware to work with
  130:   applications based on OpenSSL 0.9.7 or later. If you're using an
  131:   application based on 0.9.7 (or later) and the support you desire is only
  132:   announced for versions later than the one you need, ask the vendor to
  133:   backport their ENGINE to the version you need.
  134: 
  135:   How does "dynamic" work?
  136:   ------------------------
  137:     The dynamic ENGINE has a special flag in its implementation such that
  138:     every time application code asks for the 'dynamic' ENGINE, it in fact
  139:     gets its own copy of it. As such, multi-threaded code (or code that
  140:     multiplexes multiple uses of 'dynamic' in a single application in any
  141:     way at all) does not get confused by 'dynamic' being used to do many
  142:     independent things. Other ENGINEs typically don't do this so there is
  143:     only ever 1 ENGINE structure of its type (and reference counts are used
  144:     to keep order). The dynamic ENGINE itself provides absolutely no
  145:     cryptographic functionality, and any attempt to "initialise" the ENGINE
  146:     automatically fails. All it does provide are a few "control commands"
  147:     that can be used to control how it will load an external ENGINE
  148:     implementation from a shared-library. To see these control commands,
  149:     use the command-line;
  150: 
  151:        openssl engine -vvvv dynamic
  152: 
  153:     The "SO_PATH" control command should be used to identify the
  154:     shared-library that contains the ENGINE implementation, and "NO_VCHECK"
  155:     might possibly be useful if there is a minor version conflict and you
  156:     (or a vendor helpdesk) is convinced you can safely ignore it.
  157:     "ID" is probably only needed if a shared-library implements
  158:     multiple ENGINEs, but if you know the engine id you expect to be using,
  159:     it doesn't hurt to specify it (and this provides a sanity check if
  160:     nothing else). "LIST_ADD" is only required if you actually wish the
  161:     loaded ENGINE to be discoverable by application code later on using the
  162:     ENGINE's "id". For most applications, this isn't necessary - but some
  163:     application authors may have nifty reasons for using it. The "LOAD"
  164:     command is the only one that takes no parameters and is the command
  165:     that uses the settings from any previous commands to actually *load*
  166:     the shared-library ENGINE implementation. If this command succeeds, the
  167:     (copy of the) 'dynamic' ENGINE will magically morph into the ENGINE
  168:     that has been loaded from the shared-library. As such, any control
  169:     commands supported by the loaded ENGINE could then be executed as per
  170:     normal. Eg. if ENGINE "foo" is implemented in the shared-library
  171:     "libfoo.so" and it supports some special control command "CMD_FOO", the
  172:     following code would load and use it (NB: obviously this code has no
  173:     error checking);
  174: 
  175:        ENGINE *e = ENGINE_by_id("dynamic");
  176:        ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0);
  177:        ENGINE_ctrl_cmd_string(e, "ID", "foo", 0);
  178:        ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0);
  179:        ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0);
  180: 
  181:     For testing, the "openssl engine" utility can be useful for this sort
  182:     of thing. For example the above code excerpt would achieve much the
  183:     same result as;
  184: 
  185:        openssl engine dynamic \
  186:                  -pre SO_PATH:/lib/libfoo.so \
  187:                  -pre ID:foo \
  188:                  -pre LOAD \
  189:                  -pre "CMD_FOO:some input data"
  190: 
  191:     Or to simply see the list of commands supported by the "foo" ENGINE;
  192: 
  193:        openssl engine -vvvv dynamic \
  194:                  -pre SO_PATH:/lib/libfoo.so \
  195:                  -pre ID:foo \
  196:                  -pre LOAD
  197: 
  198:     Applications that support the ENGINE API and more specifically, the
  199:     "control commands" mechanism, will provide some way for you to pass
  200:     such commands through to ENGINEs. As such, you would select "dynamic"
  201:     as the ENGINE to use, and the parameters/commands you pass would
  202:     control the *actual* ENGINE used. Each command is actually a name-value
  203:     pair and the value can sometimes be omitted (eg. the "LOAD" command).
  204:     Whilst the syntax demonstrated in "openssl engine" uses a colon to
  205:     separate the command name from the value, applications may provide
  206:     their own syntax for making that separation (eg. a win32 registry
  207:     key-value pair may be used by some applications). The reason for the
  208:     "-pre" syntax in the "openssl engine" utility is that some commands
  209:     might be issued to an ENGINE *after* it has been initialised for use.
  210:     Eg. if an ENGINE implementation requires a smart-card to be inserted
  211:     during initialisation (or a PIN to be typed, or whatever), there may be
  212:     a control command you can issue afterwards to "forget" the smart-card
  213:     so that additional initialisation is no longer possible. In
  214:     applications such as web-servers, where potentially volatile code may
  215:     run on the same host system, this may provide some arguable security
  216:     value. In such a case, the command would be passed to the ENGINE after
  217:     it has been initialised for use, and so the "-post" switch would be
  218:     used instead. Applications may provide a different syntax for
  219:     supporting this distinction, and some may simply not provide it at all
  220:     ("-pre" is almost always what you're after, in reality).
  221: 
  222:   How do I build a "dynamic" ENGINE?
  223:   ----------------------------------
  224:     This question is trickier - currently OpenSSL bundles various ENGINE
  225:     implementations that are statically built in, and any application that
  226:     calls the "ENGINE_load_builtin_engines()" function will automatically
  227:     have all such ENGINEs available (and occupying memory). Applications
  228:     that don't call that function have no ENGINEs available like that and
  229:     would have to use "dynamic" to load any such ENGINE - but on the other
  230:     hand such applications would only have the memory footprint of any
  231:     ENGINEs explicitly loaded using user/admin provided control commands.
  232:     The main advantage of not statically linking ENGINEs and only using
  233:     "dynamic" for hardware support is that any installation using no
  234:     "external" ENGINE suffers no unnecessary memory footprint from unused
  235:     ENGINEs. Likewise, installations that do require an ENGINE incur the
  236:     overheads from only *that* ENGINE once it has been loaded.
  237: 
  238:     Sounds good? Maybe, but currently building an ENGINE implementation as
  239:     a shared-library that can be loaded by "dynamic" isn't automated in
  240:     OpenSSL's build process. It can be done manually quite easily however.
  241:     Such a shared-library can either be built with any OpenSSL code it
  242:     needs statically linked in, or it can link dynamically against OpenSSL
  243:     if OpenSSL itself is built as a shared library. The instructions are
  244:     the same in each case, but in the former (statically linked any
  245:     dependencies on OpenSSL) you must ensure OpenSSL is built with
  246:     position-independent code ("PIC"). The default OpenSSL compilation may
  247:     already specify the relevant flags to do this, but you should consult
  248:     with your compiler documentation if you are in any doubt.
  249: 
  250:     This example will show building the "atalla" ENGINE in the
  251:     crypto/engine/ directory as a shared-library for use via the "dynamic"
  252:     ENGINE.
  253:     1) "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL
  254:        source tree.
  255:     2) Recompile at least one source file so you can see all the compiler
  256:        flags (and syntax) being used to build normally. Eg;
  257:            touch hw_atalla.c ; make
  258:        will rebuild "hw_atalla.o" using all such flags.
  259:     3) Manually enter the same compilation line to compile the
  260:        "hw_atalla.c" file but with the following two changes;
  261:          (a) add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches,
  262:          (b) change the output file from "hw_atalla.o" to something new,
  263:              eg. "tmp_atalla.o"
  264:     4) Link "tmp_atalla.o" into a shared-library using the top-level
  265:        OpenSSL libraries to resolve any dependencies. The syntax for doing
  266:        this depends heavily on your system/compiler and is a nightmare
  267:        known well to anyone who has worked with shared-library portability
  268:        before. 'gcc' on Linux, for example, would use the following syntax;
  269:           gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto
  270:     5) Test your shared library using "openssl engine" as explained in the
  271:        previous section. Eg. from the top-level directory, you might try;
  272:           apps/openssl engine -vvvv dynamic \
  273:               -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD
  274:        If the shared-library loads successfully, you will see both "-pre"
  275:        commands marked as "SUCCESS" and the list of control commands
  276:        displayed (because of "-vvvv") will be the control commands for the
  277:        *atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add
  278:        the "-t" switch to the utility if you want it to try and initialise
  279:        the atalla ENGINE for use to test any possible hardware/driver
  280:        issues.
  281: 
  282:   PROBLEMS
  283:   ========
  284: 
  285:   It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32.
  286:   A quick test done right before the release showed that trying "openssl speed
  287:   -engine cswift" generated errors. If the DSO gets enabled, an attempt is made
  288:   to write at memory address 0x00000002.
  289: