Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for automatic dict and set attributes in module io #1075

Merged
merged 13 commits into from
Oct 24, 2023
Merged

Add support for automatic dict and set attributes in module io #1075

Changes from 2 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
1e956a1
First version for set, dict attributes
ncotie Oct 18, 2023
1393789
Merge pull request #1 from ncotie/io_mod_v1
ncotie Oct 18, 2023
c518569
Merge branch 'main' into main
gboeing Oct 19, 2023
22b7193
Removed comments, compressed condition clause to single check
ncotie Oct 19, 2023
e588260
Merge pull request #2 from ncotie/io_mod_v1
ncotie Oct 19, 2023
f28d8b9
Missed two trailing spaces in previous
ncotie Oct 19, 2023
2b99561
Merge pull request #3 from ncotie/io_mod_v1
ncotie Oct 19, 2023
3dd168b
After lint testing with automatic change
ncotie Oct 19, 2023
972d0e9
Merge pull request #4 from ncotie/io_mod_v1
ncotie Oct 19, 2023
7a024ac
Comment corrected
ncotie Oct 19, 2023
2e3e8af
Merge pull request #5 from ncotie/io_mod_v1
ncotie Oct 19, 2023
bdf1521
Added code testing for list, set and dict type attributes for nodes a…
ncotie Oct 20, 2023
3ff4730
Merge pull request #6 from ncotie/io_mod_v1
ncotie Oct 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion osmnx/io.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -419,8 +419,25 @@ def _convert_node_attr_types(G, dtypes=None):
Returns
-------
G : networkx.MultiDiGraph

Modified to include detection of list, dict, and set type
node attribute strings and handle their translation into
python objects of the correct type.
ncotie marked this conversation as resolved.
Show resolved Hide resolved
"""
for _, data in G.nodes(data=True):
# first, eval stringified lists, dicts or sets to convert them to objects
# edge attributes might have a single value, or a list if simplified
gboeing marked this conversation as resolved.
Show resolved Hide resolved
for attr, value in data.items():
# check for stringified lists
if value.startswith("[") and value.endswith("]"):
with contextlib.suppress(SyntaxError, ValueError):
data[attr] = ast.literal_eval(value)
# check for stringified dicts or sets: both can be converted
# to python objects using ast.literal_eval()
if value.startswith("{") and value.endswith("}"):
with contextlib.suppress(SyntaxError, ValueError):
data[attr] = ast.literal_eval(value)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We previously only ran literal_eval on edge attributes. Is there a use case for node attributes? OSMnx does not produce any node attributes of type list, dict, or set.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can describe my usage, if it makes sense to you. It's clearly all custom attributes, as I mentioned in the issue description, so not necessarily OSMnx scope.

I'm doing 15-minute city metrics, based on OSMnx with a lot of my own code on top. I fetch a walking network via OSMnx, then integrate amenity POI nodes into it, adding linking edges with lengths. The network nodes, apart from the amenities, are considered to be "residential locations", without getting into filtering details. Processing then obtains, among other things, shortest path distances, per residential node, to each amenity node. These are stored per node into a dict of dicts attribute keyed by amenity category and amenity node.
Accessible amenities are counted per residential node and edge, also, which allows for producing isochrone buffered geometries, for example.

It's true that I could have managed these outside the graph object using dicts, but I had wanted to make the work graph-oriented, and I believe it simplifies things, avoiding the need to keep separate objects consistent when doing things like truncating, etc, etc.

for attr in data.keys() & dtypes.keys():
data[attr] = dtypes[attr](data[attr])
return G
Expand All @@ -440,18 +457,28 @@ def _convert_edge_attr_types(G, dtypes=None):
Returns
-------
G : networkx.MultiDiGraph

Modified to include detection of list, dict, and set type
node attribute strings and handle their translation into
python objects of the correct type.
ncotie marked this conversation as resolved.
Show resolved Hide resolved
"""
# for each edge in the graph, eval attribute value lists and convert types
for _, _, data in G.edges(data=True, keys=False):
# remove extraneous "id" attribute added by graphml saving
data.pop("id", None)

# first, eval stringified lists to convert them to list objects
# first, eval stringified lists, dicts or sets to convert them to objects
# edge attributes might have a single value, or a list if simplified
for attr, value in data.items():
# check for stringified lists
if value.startswith("[") and value.endswith("]"):
ncotie marked this conversation as resolved.
Show resolved Hide resolved
with contextlib.suppress(SyntaxError, ValueError):
data[attr] = ast.literal_eval(value)
# check for stringified dicts or sets: both can be converted
# to python objects using ast.literal_eval()
if value.startswith("{") and value.endswith("}"):
with contextlib.suppress(SyntaxError, ValueError):
data[attr] = ast.literal_eval(value)

# next, convert attribute value types if attribute appears in dtypes
for attr in data.keys() & dtypes.keys():
Expand Down
Loading