Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ClientSession keep transferring data after closed #2497

Closed
dmmartins opened this issue Nov 10, 2017 · 2 comments
Closed

ClientSession keep transferring data after closed #2497

dmmartins opened this issue Nov 10, 2017 · 2 comments
Labels
needs-info Issue is lacking sufficient information and will be closed if not provided Stale

Comments

@dmmartins
Copy link

dmmartins commented Nov 10, 2017

Long story short

After a ClientSession is closed it keeps connected to the server and keeps transferring data.

Steps to reproduce

import asyncio
import aiohttp

async def download():
    url = 'http://example.com/large_file.iso'
    async with aiohttp.ClientSession() as http:
        async with http.get(url) as response:
            pass
    while True:
        await asyncio.sleep(1)

loop = asyncio.get_event_loop()
loop.run_until_complete(download())

When I run this code the process cpu usage goes to 90+% and the memory usage keeps increasing.
Also I can see the process is still connected to the server.

I tried explicitly closing client session but it has same behavior.

@webknjaz
Copy link
Member

@asvetlov has accidentally closed this with misleading commit reference.

@webknjaz webknjaz reopened this Nov 10, 2017
@wumpus
Copy link

wumpus commented Dec 26, 2017

The streaming interface does the right thing: if you call response.content.read(1000) and then response.close(), looks like only the first 256k of the response is read (a single read). I observed this using strace, if you're on Linux it's worth using strace instead of looking at the process size etc.

I used strace with your example and it behaved the same as the streaming interface (at least in Linux), a single system call read that got 256k of the file. Here's what it looks like with timestamps:

12:39:13 recvfrom(7, "HTTP/1.1 200 OK\r\nDate: Tue, 26 D"..., 262144, 0, NULL, NULL) = 262144
12:39:13 brk(NULL)                      = 0x1cfe000
12:39:13 brk(0x1d3f000)                 = 0x1d3f000
12:39:13 epoll_ctl(4, EPOLL_CTL_DEL, 7, 0x7ffdbdb76c20) = 0
12:39:13 epoll_wait(4, [], 1, 0)        = 0
12:39:13 epoll_wait(4, [], 1, 0)        = 0
12:39:13 close(7)                       = 0
12:39:13 epoll_wait(4, [], 1, 1000)     = 0
12:39:14 epoll_wait(4, [], 1, 0)        = 0
12:39:14 epoll_wait(4, [], 1, 1000)     = 0
12:39:15 epoll_wait(4, [], 1, 0)        = 0
12:39:15 epoll_wait(4, [], 1, 1000)     = 0
12:39:16 epoll_wait(4, [], 1, 0)        = 0
12:39:16 epoll_wait(4, [], 1, 1000)     = 0
12:39:17 epoll_wait(4, [], 1, 0)        = 0
12:39:17 epoll_wait(4, [], 1, 1000)     = 0
12:39:18 epoll_wait(4, [], 1, 0)        = 0
12:39:18 epoll_wait(4, [], 1, 1000)     = 0
12:39:19 epoll_wait(4, [], 1, 0)        = 0
12:39:19 epoll_wait(4, [], 1, 1000)     = 0
12:39:20 epoll_wait(4, [], 1, 0)        = 0
12:39:20 epoll_wait(4, [], 1, 1000)     = 0
12:39:21 epoll_wait(4, [], 1, 0)        = 0
12:39:21 epoll_wait(4, [], 1, 1000)     = 0
12:39:22 epoll_wait(4, [], 1, 0)        = 0
12:39:22 epoll_wait(4, [], 1, 1000)     = 0
12:39:23 epoll_wait(4, [], 1, 0)        = 0
12:39:23 epoll_wait(4, [], 1, 1000)     = 0
12:39:24 epoll_wait(4, [], 1, 0)        = 0
12:39:24 epoll_wait(4, 0x7fd9e74cc620, 1, 1000) = -1 EINTR (Interrupted system call)
12:39:24 --- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---

The EINTR is when I hit ^C

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-info Issue is lacking sufficient information and will be closed if not provided Stale
Projects
None yet
Development

No branches or pull requests

4 participants