asyncio_task example fails on Jupyter Notebook

[kalkschneider]

Maybe pypeln interferes with Jupyters own event loop, maybe I did something wrong. Do you have any idea?

RuntimeError: Task <Task pending coro=<_run_task() running at /opt/conda/lib/python3.7/site-packages/pypeln/asyncio_task.py:203> cb=[gather.<locals>._done_callback() at /opt/conda/lib/python3.7/asyncio/tasks.py:691]> got Future <Future pending> attached to a different loop

[cgarciae]

Can you send an example code?
I am working to resolve this for the next version but there are workarounds.

[kalkschneider]

I just used the example from your website.

from pypeln import asyncio_task as aio
import asyncio
from random import random

async def slow_add1(x):
    await asyncio.sleep(random()) # <= some slow computation
    return x + 1

async def slow_gt3(x):
    await asyncio.sleep(random()) # <= some slow computation
    return x > 3

data = range(10) # [0, 1, 2, ..., 9] 

stage = aio.map(slow_add1, data, workers = 3, maxsize = 4)
stage = aio.filter(slow_gt3, stage, workers = 2)

data = list(stage) # e.g. [5, 6, 9, 4, 8, 10, 7]

[cgarciae]

Thanks, I'll add a test for this and try to fix the issue.

[cgarciae]

Seems to be working in 0.2.1. The example changed slightly:

import pypeln as pl
import asyncio
from random import random


async def slow_add1(x):
    await asyncio.sleep(random())  # <= some slow computation
    return x + 1


async def slow_gt3(x):
    await asyncio.sleep(random())  # <= some slow computation
    return x > 3


data = range(10)  # [0, 1, 2, ..., 9]

stage = pl.task.map(slow_add1, data, workers=3, maxsize=4)
stage = pl.task.filter(slow_gt3, stage, workers=2)

data = list(stage)  # e.g. [5, 6, 9, 4, 8, 10, 7]

[kalkschneider]

Thanks a lot, it works like a charm. Btw, the new API looks a lot cleaner now.