-
Notifications
You must be signed in to change notification settings - Fork 592
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Maintaining aggregation state between task calls #6
Comments
maxmzkr
added a commit
to maxmzkr/ibis
that referenced
this issue
Nov 22, 2016
User needed in cursor
@wesm Is this issue still relevant? |
Nope |
anmyachev
added a commit
to anmyachev/ibis
that referenced
this issue
Mar 26, 2020
* Add initial support for ibis.random function (ibis-project#2060) * remove drop method (ibis-project#2114) * FEAT: OmnisciDB Supporting of cumulative ops through impala (ibis-project#2113) * FEAT: OmniSciDB - Add support for within, d_fully_within and point (ibis-project#2125) * [OmniSciDB] Fix TopK when used as filter (ibis-project#2134) * FEAT: Implement read_csv for omniscidb backend (ibis-project#2062) * FEAT: add translation rules for isnull() and notnull() for pyspark backend (ibis-project#2126) * Add non-nullable info to schema output. (ibis-project#2117) * CI: Fix PYTHONHASHSEED issue (ibis-project#2140) * FEAT: OmniSciDB Added fragment_size to table creation (ibis-project#2107) * TST: OmniSciDB - Enable logical value tests (ibis-project#2084) * CLN: Rename tutorials to keep order (ibis-project#2145) * FEAT: OmniSciDB ops.Date, ops.DateDiff and ops.TimestampDiff implementation (ibis-project#2097) Co-authored-by: James Bourbeau <[email protected]> Co-authored-by: Svetlana Emelianova <[email protected]> Co-authored-by: Ivan Ogasawara <[email protected]> Co-authored-by: Tiezheng Li <[email protected]> Co-authored-by: Marc Garcia <[email protected]>
amyskov
pushed a commit
to amyskov/ibis
that referenced
this issue
Mar 31, 2020
deepyaman
added a commit
to deepyaman/ibis
that referenced
this issue
Oct 30, 2023
# This is the 1st commit message: build: format code with `ruff format` over `black` # This is the commit message ibis-project#2: docs: fix dask compile docstring typo # This is the commit message ibis-project#3: fix(datafusion): fix some temporal operations # This is the commit message ibis-project#4: feat(datafusion): add case and if-else statements # This is the commit message ibis-project#5: chore(flake/nixpkgs): `0e18a0a0` -> `37c290f0` # This is the commit message ibis-project#6: chore(deps): update trinodb/trino docker tag to v431 # This is the commit message ibis-project#7: chore(flake/poetry2nix): `3b6124e9` -> `365879d4`
deepyaman
added a commit
to deepyaman/ibis
that referenced
this issue
Oct 30, 2023
# This is the 1st commit message: build: format code with `ruff format` over `black` # This is the commit message ibis-project#2: docs: fix dask compile docstring typo # This is the commit message ibis-project#3: fix(datafusion): fix some temporal operations # This is the commit message ibis-project#4: feat(datafusion): add case and if-else statements # This is the commit message ibis-project#5: chore(flake/nixpkgs): `0e18a0a0` -> `37c290f0` # This is the commit message ibis-project#6: chore(deps): update trinodb/trino docker tag to v431 # This is the commit message ibis-project#7: chore(flake/poetry2nix): `3b6124e9` -> `365879d4`
deepyaman
added a commit
to deepyaman/ibis
that referenced
this issue
Oct 30, 2023
# This is the 1st commit message: test(flink): turn on the common backend test suite Co-authored-by: mfatihaktas <[email protected]> # This is the commit message ibis-project#2: chore(flink): improve marker re deserializing JSON # This is the commit message ibis-project#3: test(flink): summarize the xfail reason for a test # This is the commit message ibis-project#4: refactor(flink): condense logic for table creation # This is the commit message ibis-project#5: chore(flink): mark WIP tests using notimpl instead # This is the commit message ibis-project#6: chore(flink): remove the useless/redundant reasons # This is the commit message ibis-project#7: chore(flink): change a "broken" `mark` to "notyet" # This is the commit message ibis-project#8: chore(flink): remove backend-specific expectations # This is the commit message ibis-project#9: test(flink): remove conditional expected_for_flink # This is the commit message ibis-project#10: chore(flake/nixpkgs): `0e18a0a0` -> `37c290f0` # This is the commit message ibis-project#11: chore(deps): update trinodb/trino docker tag to v431 # This is the commit message ibis-project#12: chore(flake/poetry2nix): `3b6124e9` -> `365879d4` # This is the commit message ibis-project#13: chore: constrain using expanded bounds syntax # This is the commit message ibis-project#14: chore(flake/nixpkgs): `37c290f0` -> `a9d001fd`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Issue by wesm
Tuesday Dec 09, 2014 at 20:08 GMT
Originally opened as http://github.mtv.cloudera.com/wesm/ibis/issues/6
In order to reduce serialization / data transfer costs between consecutive Update tasks. In the case of a group-wise aggregation, many aggregation states might exist simultaneously in a particular Python worker process. These can be identified by a UUID and the UUID will have to be made known to the parent Impala process
One problem with this is that we would have to keep sending tasks to the same worker process, so releasing the worker back to the pool could become problematic.
The text was updated successfully, but these errors were encountered: