Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python client implementation #1199

Merged
merged 2 commits into from
Sep 19, 2020
Merged

Add Python client implementation #1199

merged 2 commits into from
Sep 19, 2020

Conversation

sc1f
Copy link
Contributor

@sc1f sc1f commented Sep 18, 2020

This PR introduces PerspectiveClient, which mimics the Javascript client implementation by offering a fully async (asyncio or yield-based) API on top of the core Table and View public API. This is useful for testing perspective-python servers, as it offers programmatic access to Perspective's public API in Python.

PerspectiveTornadoClient is an implementation of the client API that allows users to open up remote Tables and Views on a perspective-python server, and treat them as if they were objects in the client runtime using either await or yield.

The interface is extremely simple, using async and await:

import asyncio
from perspective.tornado_handler import websocket

current_loop = asyncio.get_event_loop()
ws = websocket("ws://.../", ioloop=current_loop)

async def run():
    table = ws.open_table("data_source")
    schema = await table.schema()
    print(schema)  # {"a": "float"}
    view = table.view(row_pivots=["a"])
    data = await view.to_arrow()
    print(type(arrow)) # bytes

current_loop.run_until_complete(run())  # or current_loop.call_soon(run)

or yield-based coroutines:

from tornado import gen
from tornado.ioloop import IOLoop
from perspective.tornado_handler import websocket

current_loop = IOLoop.current()
ws = websocket("ws://.../", ioloop=current_loop)

@gen.coroutine
def run():
    table = ws.open_table("data_source")
    schema = yield table.schema()
    print(schema)  # {"a": "float"}
    view = table.view(row_pivots=["a"])
    data = yield view.to_arrow()
    print(type(arrow)) # bytes
current_loop.run_sync(run)  # or current_loop.add_callback(run)

Clean up python client, implement arrow loading, skip testing table size in remove tests for now
@sc1f sc1f self-assigned this Sep 18, 2020
@sc1f sc1f added enhancement Feature requests or improvements Python labels Sep 18, 2020
Copy link
Member

@texodus texodus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Looks great and is a super useful addition! There is a small bug I think with table names, but I will take care of fixing these in a cleanup PR.


self.send(msg)

def table(self, data, index=None, limit=None, name=str(random())):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default argument to name here is evaluated once when the method is created, so all proxy tables will have the same name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup - that is a silly mistake on my part

from .view_api import view as make_view


def table(client, data, index=None, limit=None, name=str(random())):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, and also it's probably a good idea to just make name a required arg here as this table is private to the library.

def remove_port(self):
return self._async_queue("remove_port", "table_method")

def compute(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated - but I think we can deprecate this "feature flag" - its implementation returns true for JS and Python

@@ -4,226 +4,4 @@
#
# This file is part of the Perspective library, distributed under the terms of
# the Apache License 2.0. The full license can be found in the LICENSE file.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty file

@@ -15,11 +15,13 @@ def test_remove_all(self):
tbl = Table([{"a": "abc", "b": 123}], index="a")
tbl.remove(["abc"])
assert tbl.view().to_records() == []
# assert tbl.size() == 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only adds commented blocks

@texodus
Copy link
Member

texodus commented Sep 19, 2020

Also - I believe the FOSSA check is either malfunctioning or has recently received an update to its detection criteria. This PR specifically does not introduce dependencies, so if there are legitimate failures, they are pre-existing regardless. Thus, I'm going to override this check.

@texodus texodus merged commit 4fe6e47 into master Sep 19, 2020
@texodus texodus deleted the python-client branch September 19, 2020 01:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Feature requests or improvements Python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants