Skip to content
This repository has been archived by the owner on Feb 3, 2022. It is now read-only.

Proposals binary

Jump to bottom
Robert M. Lefkowitz edited this page Jun 23, 2014 · 2 revisions

Table of Contents

DRAFT! Not yet submitted!

Proposal: Add binary 0.5.0.2 to the Haskell Platform

Proposal Author: Ian Lynagh

Maintainer: Lennart Kolmodin, Don Stewart

Introduction

This is a proposal for the binary package to be included in the next major release of the Haskell platform.

Everyone is invited to review this proposal, following the standard procedure] for proposing and reviewing packages. Review comments should be sent to the libraries mailing list by January 31st.

Credits

The following individuals contributed to the review process: <no-one,></no-one,>

Abstract

The 'binary' package provides efficient, pure binary serialisation using lazy !ByteStrings. Haskell values may be encoded to and from binary formats, written to disk as binary, or sent over the network. The binary format can either be an externally defined format, or binary's internal default format may be used if you wish only to serialise and deserialise from a Haskell program. Documentation and tarball from the hackage page: 
    http://hackage.haskell.org/package/binary
Main development repo: 
    darcs get http://code.haskell.org/binary/
Active branches: 
    darcs get http://www.haskell.org/~kolmodin/code/binary-push

    darcs get http://www.haskell.org/~kolmodin/code/binary-push-unpacked
All package requirements are met.

Rationale

`binary` provides basic functionality not yet available in the Haskell Platform. `binary` has 193 direct reverse dependencies including `Agda`, `hxt`, `Pugs` `SHA` and `tar`. It is also used by GHC, although currently GHC's copy is renamed as `binary` is not in the HP.

The API

The API is broken up into four pieces: 
 * The main interface, for serialising and deserialising values:

    http://hackage.haskell.org/packages/archive/binary/0.5.0.2/doc/html/Data-Binary.html

 * Functions for implementing serialisation for datatypes:

    http://hackage.haskell.org/packages/archive/binary/0.5.0.2/doc/html/Data-Binary-Put.html

 * Functions for implementing deserialisation for datatypes:

    http://hackage.haskell.org/packages/archive/binary/0.5.0.2/doc/html/Data-Binary-Get.html

 * An internal type used for constructing !ByteStrings incrementally:

    http://hackage.haskell.org/packages/archive/binary/0.5.0.2/doc/html/Data-Binary-Builder.html
Here is an example of the basic functionality, from the haddock docs: 
    To serialise a custom type, an instance of Binary for that type is
    required. For example, suppose we have a data structure:

    

    We can encode values of this type into bytestrings using the
    following instance, which proceeds by recursively breaking down the
    structure to serialise:

    

    Note how we write an initial tag byte to indicate each variant of the
    data type.

    We can simplify the writing of 'get' instances using monadic
    combinators:

    

    To serialise this to a bytestring, we use 'encode', which packs the
    data structure into a binary format, in a lazy bytestring

    

    Where 'v' is a binary encoded data structure. To reconstruct the
    original data, we use 'decode'

    

    The lazy !ByteString that results from 'encode' can be written to
    disk, and read from disk using Data.!ByteString.Lazy IO functions,
    such as hPutStr or writeFile:

    

    And read back with:

    

    We can also directly serialise a value to and from a Handle, or a file:

    

    And write a value to disk

Design decisions and random facts 

 * The interface is pure, modulo IO helper functions for (de)serialising directly to files pure
 * Built on top of lazy !ByteString
 * Uses CPP extension
 * When building with GHC, uses !MagicHash and !UnboxedTuple extensions
 * Uses !FlexibleContexts extension fo this instance:
   instance (Binary i, Ix i, Binary e, IArray UArray e) => Binary (UArray i e) where
 * The implementation is entirely Haskell (no additional C code or libraries).
 * The package provides a !QuickCheck testsuite and some benchmarks.
 * The package adds no new dependencies to the HP.
 * The package builds with the Simple cabal way.
 * There is no existing functionality for binary serialisation in the HP.
 * All but one exports have haddock docs, and many have complexity annotations.
 * The code is -Wall clean

Open issues 

   1. There is currently work on redesigning the parsing interface
      to support incremental parsing. The work is taking place in
      the `binary-push` and `binary-push-unpacked` branches, and
      the changes are in the `Data.Binary.Get` module.
      We may wish to accept the package with this change, rather
      than adding it in its current form.

Notes

The implementation consists of 4 modules. The modules are under 2000 lines, under 1000 of which is actual code. garbagethe standard

procedure] for proposing and reviewing packages.

Review comments should be sent to the libraries mailing list by January 31st.

Credits

The following individuals contributed to the review process: <no-one,></no-one,>

Abstract

The 'binary' package provides efficient, pure binary serialisation using lazy !ByteStrings.

Haskell values may be encoded to and from binary formats, written to disk as binary, or sent over the network.

The binary format can either be an externally defined format, or binary's internal default format may be used if you wish only to serialise and deserialise from a Haskell program.

Documentation and tarball from the hackage page: 

    http://hackage.haskell.org/package/binary

Main development repo: 

    darcs get http://code.haskell.org/binary/

Active branches: 

    darcs get http://www.haskell.org/~kolmodin/code/binary-push

    darcs get http://www.haskell.org/~kolmodin/code/binary-push-unpacked

All package requirements are met.

Rationale

`binary` provides basic functionality not yet available in the Haskell Platform.

`binary` has 193 direct reverse dependencies including `Agda`, `hxt`, `Pugs` `SHA` and `tar`. It is also used by GHC, although currently GHC's copy is renamed as `binary` is not in the HP.

The API

The API is broken up into four pieces: 

 * The main interface, for serialising and deserialising values:

    http://hackage.haskell.org/packages/archive/binary/0.5.0.2/doc/html/Data-Binary.html

 * Functions for implementing serialisation for datatypes:

    http://hackage.haskell.org/packages/archive/binary/0.5.0.2/doc/html/Data-Binary-Put.html

 * Functions for implementing deserialisation for datatypes:

    http://hackage.haskell.org/packages/archive/binary/0.5.0.2/doc/html/Data-Binary-Get.html

 * An internal type used for constructing !ByteStrings incrementally:

    http://hackage.haskell.org/packages/archive/binary/0.5.0.2/doc/html/Data-Binary-Builder.html

Here is an example of the basic functionality, from the haddock docs: 

    To serialise a custom type, an instance of Binary for that type is
    required. For example, suppose we have a data structure:






    We can encode values of this type into bytestrings using the
    following instance, which proceeds by recursively breaking down the
    structure to serialise:











    Note how we write an initial tag byte to indicate each variant of the
    data type.






    We can simplify the writing of 'get' instances using monadic
    combinators:











    To serialise this to a bytestring, we use 'encode', which packs the
    data structure into a binary format, in a lazy bytestring











    Where 'v' is a binary encoded data structure. To reconstruct the
    original data, we use 'decode'











    The lazy !ByteString that results from 'encode' can be written to
    disk, and read from disk using Data.!ByteString.Lazy IO functions,
    such as hPutStr or writeFile:











    And read back with:











    We can also directly serialise a value to and from a Handle, or a file:











    And write a value to disk












Design decisions and random facts







 * The interface is pure, modulo IO helper functions for (de)serialising directly to files pure
 * Built on top of lazy !ByteString
 * Uses CPP extension
 * When building with GHC, uses !MagicHash and !UnboxedTuple extensions
 * Uses !FlexibleContexts extension fo this instance:
   instance (Binary i, Ix i, Binary e, IArray UArray e) => Binary (UArray i e) where
 * The implementation is entirely Haskell (no additional C code or libraries).
 * The package provides a !QuickCheck testsuite and some benchmarks.
 * The package adds no new dependencies to the HP.
 * The package builds with the Simple cabal way.
 * There is no existing functionality for binary serialisation in the HP.
 * All but one exports have haddock docs, and many have complexity annotations.
 * The code is -Wall clean







Open issues







   1. There is currently work on redesigning the parsing interface
      to support incremental parsing. The work is taking place in
      the `binary-push` and `binary-push-unpacked` branches, and
      the changes are in the `Data.Binary.Get` module.
      We may wish to accept the package with this change, rather
      than adding it in its current form.







Notes






The implementation consists of 4 modules.
The modules are under 2000 lines, under 1000 of which is actual code.


              


          



  
          

            


            
Clone this wiki locally