Python-Editline
===============

A Python extension which links to libedit.so and implements the
shell-completion functionality completely independent of
libreadline.so and readline.

My operational goal was originally to try to get the readline
functionality of libedit to "just work". Apparently me, and a bazillion
others have tried and only got something marginally functional.

I changed the plan. For reasons I do not know, the 'readline' interface
of Python is heavily favouring the libreadline.so interface. In my view,
that is a problem on several fronts. It makes portability to libedit a
pain, and have you seen how many freeking global variable libreadline
uses just to function? It is crazy.

My (so far successful) approach has been to abandon the readline api and
work towards a generic line-editor interface to which both libedit (and
readline if necessary) will conform.

**The goals are:**

-  to create a Python installation devoid of GPL
-  run correctly on Linux, \*BSD, SunOS and Mac
-  not muck up the current terminal
-  use libedit either built-in on various BSDs or use
   http://thrysoee.dk/editline/
-  push it back into Python main-line
-  Move the bulk of the "completion" functionality back into Python code
   where it is a hell of a lot easier to manage lists of strings

Current State
-------------

I have done various tries at this. The module/ directory is so far the
one that works.

-  Basic tab completion runs smoothly (... in Python code)
-  Enhanced features of tab-completion

    - Across lists (index completion and view)
    - Across dicts (key-completion and view)
    - Support for tab-completion in any complexity of command

-  interface starts up automatically with 'python -i'
-  system-wide editline instance is created automatically
-  terminal resizing works
-  support for RIGHT-SIDE prompt (as well as std) is present (I don't
   think readline can even do that)
-  support for ^c and ^d match the readline functionality
-  no terminal changes occur after running
-  history support works

   -  adding commands
   -  loading history file
   -  saving history file

-  setup.py will construct either an extension which refers to
   libedit.so or will build a larger extension with the libedit sources
   embedded.
-  support for "history" and bash-like shortcuts to access historic cmds
-  significant test cases have been implemented


Installation
------------

Simple Case -- No sitecustomize.py in python or virtual-env
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To get this thing working, I recommend this:

.. code:: bash

    # cd /handy/location
    # git clone https://github.com/mark-nicholson/python-editline.git
    # make 
    # python3 -mvenv myvenv
    # . myvenv/bin/activate
    (myvenv) # python3 setup.py install 
    (myvenv) # python
    >>>

The 'gnu make' is there to download and prep the libedit distribution.

More Complicated Situation
~~~~~~~~~~~~~~~~~~~~~~~~~~

The installer has support to be able to work-around and merge (sort of)
the pre-existing sitecustomize.py file and the new necessary changes.

.. code:: bash

    # cd /handy/location
    # git clone https://github.com/mark-nicholson/python-editline.git
    # make 
    # python3 -mvenv myvenv
    # . myvenv/bin/activate
    (myvenv) # python3 setup.py build_py --merge-sitecustomize
    (myvenv) # python3 setup.py install
    (myvenv) # python3
    >>>

The second situation will create a backup of the original
sitecustomize.py insitu.

A note of caution before you pull all of your hair out. SOME
distributions (pointing at you Ubuntu) drop in a sitecustomize.py file
in /usr/lib/python... which is EXTREMELY annoying as it completely
prevents a VirtualENV from creating its own 'customization' using this
because the system path is always ahead of the venv path. sigh. You can
hack this by renaming it to usercustomize.py and dropping it at the same
place -- ugly but effective.

I tested this in both a default install and virtual-env. The
instructions detail the VENV method which I used most. The
sitecustomize.py file is critical as it is the init-hook which
disengages readline as the default completer and installs editline.

Another way to make it work in a default install, you can edit site.py
and replace the enablerlcompleter() function with the
enable\_line\_completer() function from sitecustomize.py in the repo. It
does not matter whether you do it in the cpython source tree, or after
it is installed. It is coded in such a way that if editline support is
absent, it will still fall back to readline.

Situation - I've got a bork'd libedit, how do I force the builtin?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For this case, the default install *will hopefully* figure out that
``libedit`` is bork'd and all is well. If it doesn't, then you'll have
to manually force it.

.. code:: bash

    # cd /handy/location
    # git clone https://github.com/mark-nicholson/python-editline.git
    # make 
    # python3 -mvenv myvenv
    # . myvenv/bin/activate
    (myvenv) # python3 setup.py build --builtin-libedit
    (myvenv) # python3 setup.py install
    (myvenv) # python3
    >>>

The ``--builtin-libedit`` option to the ``build`` phase overrides the
system snooping. It will still do some checks to ensure it has enough
stuff available to build the builtin-libedit (and link it).

How do I check?
^^^^^^^^^^^^^^^

    >>> import _editline
    >>> gi = _editline.get_global_instance()
    >>> gi.rprompt = '<RP'
    >>>                                                              <RP
    >>>

Get readline to do THAT! (ok, if your browser is narrow, you should have
the '<RP' on the other end of the command line -- the interpreter should
do it right)

Premise? Ok, so the editline infra uses instances, NOT GLOBALS. The
get\_global\_instance() API gives you "roughly" the same thing as
'import readline'. We will see if we can smooth it out a bit more later.

Annoyances
~~~~~~~~~~

I am vexed and cannot understand why Modules/main.c in the Python build
does this

.. code:: C

    v = PyImport_ImportModule("readline");

I've commented it out and found no alteration of functionality. It
*probably* is some legacy code which was put in in the dark-ages, and
just doesn't piss anyone off.... except me. Its presences *seems*
benign, but it still concerns me. Even when, disconcertingly, both
readline.so is loaded (by main.c) and \_editline.so is loaded by the
standard method, this then has both libreadline.so and libedit.so loaded
into the process. There will certainly be a symbol-collision for
'readline()' (in C) as both have it. I suspect that it will work without
any issues since *editline.so only calls the libedit API (el*\ \*) and
does not reference the C call to readline() in libedit. Tread carefully.

If you are rolling your own Python interpreter, patch out the silly
import in main.c. Then, even if you have both readline.so and
\_editline.so, they will be mutually exclusive! (Check the patches/
directory.)


Tasks
-----

The baseline code is working. It is considerably simpler than the
readline implementation and also uses instances (even the 'system' one)
so it can support multiple instances per interpreter.

-  Create changeset for Python codebase (integrate everything there?)
-  Implement more test infrastructure to beat it like a rented mule
-  build out functionality for

   -  bind()
   -  el\_get()
   -  el\_set()
   -  history()

-  create a mapping routine so 'parse-and-bind' can support a common
   command format and translate to the "local" flavour.
-  potentially create a subclass which is a readline
   drop-in-replacement.


Acknowledgements
----------------

Thank you to:

* http://thrysoee.dk/editline/ for creating the portable version of the NetBSD 'libedit' library.
* the libedit developers who made the examples/ directory. Great baseline code.