DOC: Add DataFrame.index.levels #55437

shiersansi · 2023-10-07T14:08:15Z

Add docstrings for MultiIndex.levels and MultiIndex.codes

datapythonista

I think you misunderstood the list on the description. You shouldn't ignore the existing items in the list, and add one of what you did. You need to do what the existing items say, and check them to show that you did the relevant points. If you see the CI, you'll see there are error, that you could have detected earlier with the tasks in that list. No big deal having run those things locally, but they'll be helpful to prevent the CI being red, which prevents us from merging this.

pandas/core/indexes/multi.py

datapythonista · 2023-10-07T20:13:57Z

pandas/core/indexes/multi.py

-        # create new IndexEngine
-        # https://github.com/pandas-dev/pandas/issues/31648
+        """
+        Returns a tuple of Index objects representing the levels


I think the short summary needs to fit in one line. I wouldn't go into technical details like if the return is a tuple. The best I can think of as a short summary is Levels of the MultiIndex.. Then in the following paragraph I would explain what are levels, what they mean conceptually, how they can be seen in pandas...

datapythonista · 2023-10-07T20:15:54Z

pandas/core/indexes/multi.py

+        Each level is an Index object containing unique values
+        from that level of the MultiIndex.
+
+        Returns


In this case this is an attribute (MultiIndex.levels not MultiIndex.levels()), so for the users it doesn't return anything, it contains the value, and this section is not expected to exist. I'm not sure how we do for other attributes in the docs, they can be a good reference.

pandas/core/indexes/multi.py

datapythonista

Thanks for the work on this @shiersansi, amazing work, I really like your explanations here, they are super clear.

I added some comments that I think can make the examples better, but outstanding work, thanks for it.

datapythonista · 2023-10-08T07:48:06Z

pandas/core/indexes/multi.py

+        """
+        Levels of the MultiIndex.
+
+        What are levels?


I'd prefer to remove the questions. In isolation I think they are fine, but since we don't use them in any other API documentation page, they feel a bit strange and inconsistent. I think it's easy to understand the explanations without the questions, so no big deal.

pandas/core/indexes/multi.py

datapythonista · 2023-10-08T07:54:51Z

pandas/core/indexes/multi.py

+
+        Examples
+        --------
+        Example 1:


I think you can remove the Example 1 and just show the example directly.

datapythonista · 2023-10-08T07:55:32Z

pandas/core/indexes/multi.py

+        FrozenList([[0, 1, 2], ['green', 'purple']])
+
+        Example 2:
+        >>> idx = pd.MultiIndex.from_product([['John', 'Josh', 'Alex'],


Can you reuse the index from above, so this is less verbose please.

datapythonista · 2023-10-08T07:56:24Z

pandas/core/indexes/multi.py

+        Example 2:
+        >>> idx = pd.MultiIndex.from_product([['John', 'Josh', 'Alex'],
+        ... list('abcde')], names=['Person', 'Letter'])
+        >>> large = pd.DataFrame(data=np.random.randn(15, 2),


I think it's better to just call this df instead of large

datapythonista · 2023-10-08T08:03:26Z

pandas/core/indexes/multi.py

+        ... list('abcde')], names=['Person', 'Letter'])
+        >>> large = pd.DataFrame(data=np.random.randn(15, 2),
+        ... index=idx, columns=['one', 'two'])
+        >>> small = large.loc[['Jo'==d[0:2] for d in


Can you show the dataframe here first, so the user can better understand the data of the example? I think you'll want to avoid using random data in the example, as it'll make your life more difficult. And if you use data that is meaningful instead of some random things, that will also help users understand better. As an idea, you can use a MultiIndex of animals with class/animal (e.g. insect/ant, insect/spides, mammal/goat...) and use a dataframe with just one column, for example number of legs (we use this example in other docs). The example is silly, but meaningful and any user can understand it very quickly, and focus on the concepts, not in understanding the data.

Then, as said, you can show the dataframe first, show the levels of the multiindex from it (i.e. df.index.levels), and then you filter the data many_leg_animals = animals[animals.num_legs > 4] to finally show that even if now there are no mammals, in the data, the multiindex still contains all the levels.

Thank you for your suggestion. Is it easier to understand the example part?

I have a problem that I may need your help with. When I tested the code after I committed it, four tests failed, but I couldn't figure out exactly what went wrong from the error log provided.

datapythonista

Thanks @shiersansi, this is getting better, but I still have some comments, as I find the examples a bit confusing. Let me know if you have any question or comment.

For the examples, you should make sure PEP-8 is respected (like having one spaces after commas...). I can show you more in detail where to see the errors in the next iteration, when you address the comments.

Thanks again for working on this!

datapythonista · 2023-10-09T08:15:18Z

pandas/core/indexes/multi.py

+
+        Examples
+        --------
+        >>> idx = pd.MultiIndex.from_product([('insect', 'mammal'),


If you do from_product we'll have all the options, like insect/goat, mammal/spider, which we don't want. There are multiple options, one is just to create a "normal" dataframe with all columns, and then set the index to create the MultiIndex. That may be the simpler, but if you find another that is simpler or as clear, feel free to use any other way. But data should be something like:

mammal - goat - 4 mammal - human - 2 insect - ant - 6 insect - spider - 8

Not things like insect - goat - 1 which I think it'll just confuse users, as the data doesn't make sense.

datapythonista · 2023-10-09T08:18:45Z

pandas/core/indexes/multi.py

+
+        If the DataFrame not using all levels, this method still returns the original
+        levels of the MultiIndex.
+        When using this method on a DataFrame index, it may show "extra".


Is this last sentence talking about the MultiIndex returning all original levels? I personally find the it may show extra comment misleading. If you want to complement the previous sentence, what about something like For example, if a MultiIndex is created with levels A, B, C, and the DataFrame using it filters out all rows of the level C, MultiIndex.levels will still return A, B, C. Something like this should make things easier to understand than this last comment IMHO.

datapythonista · 2023-10-09T08:19:44Z

pandas/core/indexes/multi.py

+        --------
+        >>> idx = pd.MultiIndex.from_product([('insect', 'mammal'),
+        ... ('goat', 'dog','cat','ant','spides')], names=['class', 'animal'])
+        >>> print(idx.levels)


I'd better show the whole dataframe instead when it's ready, and you don't need the print, just >>> df or >>> leg_num.

datapythonista · 2023-10-09T08:21:35Z

pandas/core/indexes/multi.py

+        >>> leg_num = pd.DataFrame(data=(1,2,3,4,5,6,7,8,9,2),
+        ... index=idx, columns=['leg'])
+        >>> insect_leg_num = leg_num.loc[['insect'==d for d in
+        ... leg_num.index.get_level_values('class')]]


I don't get this, why not something simpler like large_leg_num = leg_num[leg_num.num_legs > 4] or something simpler where the user doesn't need to put a log of effort to understand something that is not what this docstring is about?

datapythonista · 2023-10-09T08:23:23Z

pandas/core/indexes/multi.py

+        >>> insect_leg_num = leg_num.loc[['insect'==d for d in
+        ... leg_num.index.get_level_values('class')]]
+
+        >>> print(leg_num.index.levels)


I'd show this as soon as the dataframe is ready, then continue with the filtering and showing that the levels didn't change. You don't need the print here either. And I think it'd be good to add a comment before you show the filtering, to let users know what you're going to show them, which is not obvious.

I have revised the doc content again and look forward to your new suggestions

datapythonista

Thanks @shiersansi, looking better. I think we should show the data so the examples are clearer, and added few more thoughts, but looks good in general.

pandas/core/indexes/multi.py

datapythonista · 2023-10-11T17:09:56Z

pandas/core/indexes/multi.py

+        FrozenList([['mammal'], ['cat', 'dog', 'goat', 'human']])
+
+        If the number of Legs is greater than 2 as the filter condition,
+        the levels do not change


This is not clear to me, sorry. I think something like MultiIndex levels will not change even if the DataFrame using the MultiIndex does not contain all them anymore:

pandas/core/indexes/multi.py

datapythonista · 2023-10-11T17:14:28Z

pandas/core/indexes/multi.py

+        level in the MultiIndex and contains the unique values found in that
+        specific level.
+
+        if a MultiIndex is created with levels A, B, C, and the DataFrame using


Suggested change

if a MultiIndex is created with levels A, B, C, and the DataFrame using

If a MultiIndex is created with levels A, B, C, and the DataFrame using

Feels a bit strange to jump into an example case before explaining the concept, but I think it's easier to understand this way.

datapythonista · 2023-10-12T08:04:41Z

pandas/core/indexes/multi.py

+        FrozenList([['mammal'], ['cat', 'dog', 'goat', 'human']])
+
+        MultiIndex levels will not change even if the DataFrame using the MultiIndex
+        does not contain all them anymore:


I think this can make it even more clear (and I think it's better to leave a blank line between the comment and the code. But other than this small suggestion seems perfect, I think this is a very great addition to the docs.

Suggested change

does not contain all them anymore:

does not contain all them anymore. See How "human" is not in the DataFrame, but it is still in levels:

datapythonista · 2023-10-12T08:05:00Z

/preview

github-actions · 2023-10-12T08:05:11Z

Website preview of this PR available at: https://pandas.pydata.org/preview/55437/

datapythonista · 2023-10-12T08:06:41Z

Looks like something is broken in the last part of the examples, you can see it here: https://pandas.pydata.org/preview/55437/docs/reference/api/pandas.MultiIndex.levels.html#pandas.MultiIndex.levels

Can you have a look please?

pandas/core/indexes/multi.py

datapythonista

Amazing work @shiersansi, thanks a lot for working on this!

shiersansi · 2023-10-12T09:45:26Z

Amazing work @shiersansi, thanks a lot for working on this!

Thank you for your help and advice. Your help has made me understand more about how to contribute to open source projects, and I have learned a lot through this docstring modification

datapythonista · 2023-10-12T09:48:45Z

Amazing work @shiersansi, thanks a lot for working on this!

Thank you for your help and advice. Your help has made me understand more about how to contribute to open source projects, and I have learned a lot through this docstring modification

Amazing, I'm glad to hear. Feel free to continue working in other things, and please let me know if you need help. For this PR we'll wait until the CI finishes, and another core developer has time to review it, in case I missed something, but it should be ready from your side.

mroeschke · 2023-10-12T15:11:07Z

Thanks @shiersansi

shiersansi added 3 commits October 7, 2023 21:42

modified: pandas/core/indexes/multi.py

50ff6c2

modified: pandas/core/indexes/multi.py

f5b7e29

modified: pandas/core/indexes/multi.py

2c8b861

datapythonista reviewed Oct 7, 2023

View reviewed changes

datapythonista added Docs Indexing Related to indexing on series/frames, not to indexes themselves labels Oct 7, 2023

shiersansi added 6 commits October 8, 2023 11:59

modified: pandas/core/indexes/multi.py

6241262

modified: pandas/core/indexes/multi.py

aac99e1

modified: pandas/core/indexes/multi.py

a3984e1

modified: pandas/core/indexes/multi.py

c11f34a

modified: pandas/core/indexes/multi.py

f2bf96a

modified: pandas/core/indexes/multi.py

164df6f

datapythonista reviewed Oct 8, 2023

View reviewed changes

datapythonista changed the title ~~Doc: add DataFrame.index.levels~~ DOC: Add DataFrame.index.levels Oct 8, 2023

shiersansi added 4 commits October 8, 2023 16:53

modified: pandas/core/indexes/multi.py

14be467

modified: pandas/core/indexes/multi.py

f5085f8

modified: pandas/core/indexes/multi.py

1bca09d

modified: pandas/core/indexes/multi.py

ad66dfb

datapythonista reviewed Oct 9, 2023

View reviewed changes

shiersansi added 5 commits October 9, 2023 20:40

modified: pandas/core/indexes/multi.py

9ea6ebb

modified: pandas/core/indexes/multi.py

329e45c

modified: ../pandas/core/indexes/multi.py

b871123

modified: ../pandas/core/indexes/multi.py

ca13b9a

modified: pandas/core/indexes/multi.py

4865b2a

datapythonista reviewed Oct 11, 2023

View reviewed changes

shiersansi added 5 commits October 12, 2023 09:18

modified: pandas/core/indexes/multi.py

6eaa176

modified: pandas/core/indexes/multi.py

ed8ec94

modified: pandas/core/indexes/multi.py

39eacd2

modified: pandas/core/indexes/multi.py

19298c4

modified: pandas/core/indexes/multi.py

324b682

shiersansi added 3 commits October 12, 2023 13:31

modified: pandas/core/indexes/multi.py

1e1c873

modified: pandas/core/indexes/multi.py

1d658e0

modified: pandas/core/indexes/multi.py

efe5e0a

datapythonista reviewed Oct 12, 2023

View reviewed changes

modified: pandas/core/indexes/multi.py

596d55c

datapythonista reviewed Oct 12, 2023

View reviewed changes

pandas/core/indexes/multi.py Outdated Show resolved Hide resolved

Update pandas/core/indexes/multi.py

786abee

datapythonista approved these changes Oct 12, 2023

View reviewed changes

mroeschke approved these changes Oct 12, 2023

View reviewed changes

mroeschke added this to the 2.2 milestone Oct 12, 2023

mroeschke merged commit 7ef617e into pandas-dev:main Oct 12, 2023
33 checks passed

chethanc1011 mentioned this pull request Nov 30, 2023

DOC: Added Documentation for MultiIndex.codes #56265

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOC: Add DataFrame.index.levels #55437

DOC: Add DataFrame.index.levels #55437

shiersansi commented Oct 7, 2023 •

edited by datapythonista

Loading

datapythonista left a comment

datapythonista Oct 7, 2023

datapythonista Oct 7, 2023

datapythonista left a comment

datapythonista Oct 8, 2023

datapythonista Oct 8, 2023

datapythonista Oct 8, 2023

datapythonista Oct 8, 2023

datapythonista Oct 8, 2023

shiersansi Oct 8, 2023

shiersansi Oct 8, 2023

datapythonista left a comment

datapythonista Oct 9, 2023

datapythonista Oct 9, 2023

datapythonista Oct 9, 2023

datapythonista Oct 9, 2023

datapythonista Oct 9, 2023

shiersansi Oct 11, 2023

datapythonista left a comment

datapythonista Oct 11, 2023

datapythonista Oct 11, 2023

datapythonista Oct 12, 2023

datapythonista commented Oct 12, 2023

github-actions bot commented Oct 12, 2023

datapythonista commented Oct 12, 2023

datapythonista left a comment

shiersansi commented Oct 12, 2023

datapythonista commented Oct 12, 2023

mroeschke commented Oct 12, 2023

	if a MultiIndex is created with levels A, B, C, and the DataFrame using
	If a MultiIndex is created with levels A, B, C, and the DataFrame using

	does not contain all them anymore:
	does not contain all them anymore. See How "human" is not in the DataFrame, but it is still in levels:

DOC: Add DataFrame.index.levels #55437

DOC: Add DataFrame.index.levels #55437

Conversation

shiersansi commented Oct 7, 2023 • edited by datapythonista Loading

datapythonista left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

datapythonista left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

datapythonista left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

datapythonista left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

datapythonista commented Oct 12, 2023

github-actions bot commented Oct 12, 2023

datapythonista commented Oct 12, 2023

datapythonista left a comment

Choose a reason for hiding this comment

shiersansi commented Oct 12, 2023

datapythonista commented Oct 12, 2023

mroeschke commented Oct 12, 2023

shiersansi commented Oct 7, 2023 •

edited by datapythonista

Loading