Pyrolite - Python Remote Objects "light" and Pickle for Java/.NET

  Pyrolite is written by Irmen de Jong ([email protected]).
  This software is distributed under the terms written in the file `LICENSE`.


Contents:
    1. INTRODUCTION
    2. THE LIBRARY
    3. TYPE MAPPINGS
    4. EXCEPTIONS
    5. SECURITY WARNING
    6. RECOMMENDED DEPENDENCY: SERPENT SERIALIZER
    7. DOWNLOAD COMPILED BINARIES


1. INTRODUCTION
---------------------

This library allows your Java or .NET program to interface very easily with
the Python world. It uses the Pyro protocol to call methods on remote objects.
(See https://github.com/irmen/Pyro4). To that end, it contains and uses a
feature complete pickle protocol implementation -read and write- to exchange
data with Pyro/Python.

Pyrolite only implements part of the client side Pyro library, hence its name
'lite'...  But because Pyrolite has no dependencies, it is a much lighter way
to use Pyro from Java/.NET than a solution with jython+pyro or IronPython+Pyro
would provide. So if you don't need Pyro's full feature set, and don't require
your Java/.NET code to host Pyro objects itself, Pyrolite may be a good choice
to connect java or .NET and python.


Java packages:   net.razorvine.pickle,  net.razorvine.pyro
.NET namespaces: Razorvine.Pickle, Razorvine.Pyro

Small piece of example code in Java:

    import net.razorvine.pyro.*;
    
    NameServerProxy ns = NameServerProxy.locateNS(null);
    PyroProxy remoteobject = new PyroProxy(ns.lookup("Your.Pyro.Object"));
    Object result = remoteobject.call("pythonmethod", 42, "hello", new int[]{1,2,3});
    String message = (String)result;  // cast to the type that 'pythonmethod' returns
    System.out.println("result message="+message);
    remoteobject.close();
    ns.close();
    
Same piece of example code in C#:

    using Razorvine.Pyro;
    
    using( NameServerProxy ns = NameServerProxy.locateNS(null) )
    {
        // this uses the statically typed proxy class:
        using( PyroProxy something = new PyroProxy(ns.lookup("Your.Pyro.Object")) )
        {
            object result = something.call("pythonmethod", 42, "hello", new int[]{1,2,3});
            string message = (string)result;  // cast to the type that 'pythonmethod' returns
            Console.WriteLine("result message="+message);
            result = something.getattr("remote_attribute");
            Console.WriteLine("remote attribute="+result);
        }
        
        // but you can also use it as a dynamic!
        using( dynamic something = new PyroProxy(ns.lookup("Your.Pyro.Object")) )
        {
            object result = something.pythonmethod(42, "hello", new int[]{1,2,3});
            string message = (string)result;  // cast to the type that 'pythonmethod' returns
            Console.WriteLine("result message="+message);
            result = something.remote_attribute;
            Console.WriteLine("remote attribute="+result);
        }
    }
        
More examples can be found in the examples directory. You could also study the
unit tests. These include a lot of code dealing with just the pickle subsystem
as well.


2. THE LIBRARY
---------------------

The library consists of 2 parts: a thin version of the client side part of
Pyro, and a feature complete implementation of Python's pickle protocol,
including memoization. It is fully compatible with pickles from Python 2.x and
Python 3.x, and you can use it idependently from the rest of the library, to
read and write Python pickle structures.

Pickle protocol version support: reading: 0,1,2,3,4; writing: 2.
Pyrolite can read all pickle protocol versions  (0 to 4, so this includes
the latest additions made in Python 3.4).
Pyrolite always writes pickles in protocol version 2. There are no plans on 
including protocol version 1 support. Protocols 3 and 4 contain some nice new
features which may eventually be utilized, but for now, only version 2 is used.


The source archive contains the full source, and also unit test code and a
couple of example programs in the java/test/ directory.

Pyrolite speaks Pyro4 protocol version 48 only (Pyro 4.38 and later).
(get an older version of Pyrolite if you need to connect to earlier Pyro versions) 
The java library requires java 8 (jdk/jre 1.8) or newer to compile and run.
The .net library requires .net framework 4.0 or newer.
The Java code was developed using Eclipse.
The C#/.NET source was developed using mono, monodevelop and sharpdevelop.



3. TYPE MAPPINGS
---------------------

Pyrolite does the following type mappings:

PYTHON    ---->     JAVA
------              ----
None                null
bool                boolean
int                 int
long                long or BigInteger  (depending on size)
string              String
unicode             String
complex             net.razorvine.pickle.objects.ComplexNumber
datetime.date       java.util.Calendar
datetime.datetime   java.util.Calendar
datetime.time       net.razorvine.pickle.objects.Time
datetime.timedelta  net.razorvine.pickle.objects.TimeDelta
float               double   (float isn't used) 
array.array         array of appropriate primitive type (char, int, short, long, float, double)
list                java.util.List<Object>
tuple               Object[]
set                 java.util.Set
dict                java.util.Map
bytes               byte[]
bytearray           byte[]
decimal             BigDecimal    
custom class        Map<String, Object>  (dict with class attributes including its name in "__class__")
Pyro4.core.URI      net.razorvine.pyro.PyroURI
Pyro4.core.Proxy    net.razorvine.pyro.PyroProxy
Pyro4.errors.*      net.razorvine.pyro.PyroException
Pyro4.utils.flame.FlameBuiltin     net.razorvine.pyro.FlameBuiltin 
Pyro4.utils.flame.FlameModule      net.razorvine.pyro.FlameModule 
Pyro4.utils.flame.RemoteInteractiveConsole    net.razorvine.pyro.FlameRemoteConsole 

The unpickler simply returns an Object. Because Java is a statically typed
language you will have to cast that to the appropriate type. Refer to this
table to see what you can expect to receive.
                    

JAVA     ---->      PYTHON
-----               ------
null                None
boolean             bool
byte                int
char                str/unicode (length 1)
String              str/unicode
double              float
float               float
int                 int
short               int
BigDecimal          decimal
BigInteger          long
any array           array if elements are primitive type (else tuple)
Object[]            tuple (cannot contain self-references)
byte[]              bytearray
java.util.Date      datetime.datetime
java.util.Calendar  datetime.datetime
java.sql.Date       datetime.date
java.sql.Time       datetime.time
java.sql.Timestamp  datetime.datetime
Enum                the enum value as string
java.util.Set       set
Map, Hashtable      dict
Vector, Collection  list
Serializable        treated as a JavaBean, see below.
JavaBean            dict of the bean's public properties + __class__ for the bean's type.
net.razorvine.pyro.PyroURI      Pyro4.core.URI
net.razorvine.pyro.PyroProxy    cannot be pickled.


PYTHON      ---->    C#
------              ----
None                null
bool                bool
int                 int
long                long (c# doesn't have BigInteger so there's a limit on the size)
string              string
unicode             string
complex             Razorvine.Pickle.Objects.ComplexNumber
datetime.date       DateTime
datetime.datetime   DateTime
datetime.time       TimeSpan
datetime.timedelta  TimeSpan
float               double
array.array         array (all kinds of element types supported)
list                ArrayList (of objects)
tuple               object[]
set                 HashSet<object>
dict                Hashtable (key=object, value=object)
bytes               ubyte[]
bytearray           ubyte[]
decimal             decimal
custom class        IDictionary<string, object>  (dict with class attributes including its name in "__class__")
Pyro4.core.URI      Razorvine.Pyro.PyroURI
Pyro4.core.Proxy    Razorvine.Pyro.PyroProxy
Pyro4.errors.*      Razorvine.Pyro.PyroException
Pyro4.utils.flame.FlameBuiltin     Razorvine.Pyro.FlameBuiltin 
Pyro4.utils.flame.FlameModule      Razorvine.Pyro.FlameModule 
Pyro4.utils.flame.RemoteInteractiveConsole    Razorvine.Pyro.FlameRemoteConsole 

The unpickler simply returns an object. Because C# is a statically typed
language you will have to cast that to the appropriate type. Refer to this
table to see what you can expect to receive. TIP: if you are using C# 4.0 you
can use the 'dynamic' type in some places to avoid excessive type casting.


  C#      ---->     PYTHON
------              -------
null                None
boolean             bool
byte                byte
sbyte               int
char                str/unicode (length 1)
string              str/unicode
double              float
float               float
int/short/sbyte     int
uint/ushort/byte    int
decimal             decimal
byte[]              bytearray
primitivetype[]     array
object[]            tuple  (cannot contain self-references)
DateTime            datetime.datetime
TimeSpan            datetime.timedelta
Enum                just the enum value as string
HashSet             set
Map, Hashtable      dict
Collection          list
Enumerable          list
object with public properties      dictionary of those properties + __class__
anonymous class type        dictonary of the public properties
Razorvine.Pyro.PyroURI      Pyro4.core.URI
Razorvine.Pyro.PyroProxy    cannot be pickled.


4. EXCEPTIONS
---------------------

Pyrolite also maps Python exceptions that may occur in the remote object. It
has a rather simplistic approach:

*all* exceptions, including the Pyro ones (Pyro4.errors.*), are converted to
PyroException objects. PyroException is a normal Java or C# exception type,
and it will be thrown as a normal exception in your program.  The message
string is taken from the original exception. The remote traceback string is
available on the PyroException object in the _pyroTraceback field.


5. SECURITY WARNING
---------------------

If you use Pyrolite to talk to a Pyro server it will use pickle as
serialization protocol. THIS MEANS YOUR PYRO SERVER CAN BE VULNERABLE TO
REMOTE ARBITRARY CODE EXECUTION (because of the well known security problem
with the pickle protocol).

The current version of Pyrolite is only able to talk to Pyro when using the
pickle protocol. Because pickle is not enabled by default in recent Pyro
versions, you will have to configure Pyro to allow the use of pickle. See the
Pyro documentation on how to do this. A future Pyrolite version may improve
this by allowing other serializers.

Note: your .NET or Java client code is perfectly safe. The unpickler
implementation in Pyrolite doesn't randomly construct arbitrary objects and is
safe to use for parsing data from the network.


6. RECOMMENDED DEPENDENCY FOR PYRO: SERPENT SERIALIZER
------------------------------------------------------

The default serializer is set to serpent. Unless you change the configuration
to use pickle instead, Pyrolite will require the Razorvine.Serpent assembly or
the serpent jar to be available. If you do not supply this library, Pyrolite
will still work but only with the built-in pickle serializer. Serpent is a
separate project, and the library is not included in the Pyrolite project.

You can find the Serpent project at: https://github.com/irmen/Serpent
You need version 1.16 of Serpent, or newer.


7. DOWNLOAD COMPILED LIBRARY
----------------------------

Precompiled libraries are available in the usual sources.

For .NET you can get it with nuget; https://www.nuget.org/packages/Razorvine.Pyrolite/
For Java you can get it from Maven central, groupid `net.razorvine` artifactid `pyrolite`.